-- Logs begin at Tue 2022-05-24 13:11:55 UTC, end at Tue 2022-05-24 14:21:26 UTC. -- May 24 13:40:38.537414 dv-cinder-842129-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 24 13:40:38.546288 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 13:40:38 2022] *** May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: nodename: dv-cinder-842129-1-iscsi May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: machine: x86_64 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: clock source: unix May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: pcre jit disabled May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: detected number of CPU cores: 4 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: current working directory: / May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: detected binary path: /usr/bin/uwsgi-core May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: your processes number limit is 47680 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: your memory page size is 4096 bytes May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: detected max file descriptor number: 2048 May 24 13:40:38.548316 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: lock engine: pthread robust mutexes May 24 13:40:38.549055 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: thunder lock: enabled May 24 13:40:38.549055 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 24 13:40:38.549055 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 13:40:38.602084 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: Python main interpreter initialized at 0x55e9489fa9c0 May 24 13:40:38.602084 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: python threads support enabled May 24 13:40:38.602084 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: your server socket listen backlog is limited to 100 connections May 24 13:40:38.602084 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: your mercy for graceful operations on workers is 90 seconds May 24 13:40:38.602458 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: mapped 403077 bytes (393 KB) for 2 cores May 24 13:40:38.602529 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: *** Operational MODE: preforking *** May 24 13:40:38.602782 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: *** uWSGI is running in multiple interpreter mode *** May 24 13:40:38.602782 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: spawned uWSGI master process (pid: 80156) May 24 13:40:38.602989 dv-cinder-842129-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 24 13:40:38.604015 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: spawned uWSGI worker 1 (pid: 80166, cores: 1) May 24 13:40:38.604015 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: spawned uWSGI worker 2 (pid: 80167, cores: 1) May 24 13:40:38.604550 dv-cinder-842129-1-iscsi devstack@keystone.service[80156]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 13:40:44.263701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 13:40:44.263701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 24 13:40:44.264540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 13:40:44.264540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 24 13:40:44.302012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 13:40:44.302012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes May 24 13:40:44.302516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 13:40:44.302516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: from cryptography.utils import int_from_bytes May 24 13:40:44.314293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 13:40:44.314500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 13:40:44.314695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] command line args: [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 13:40:44.314966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 13:40:44.315147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ================================================================================ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 13:40:44.315403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.315653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_dir = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.315871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.316063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_source = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.316250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] debug = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.316476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.317894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_config_append = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.318319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.318319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_dir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.318451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.318636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.318871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.319120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.319369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.319551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.319732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.320002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.320255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.320492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.320729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.320964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.321199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.321450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.321699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.321976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.322482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.322671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] publish_errors = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.322907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.323140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.323322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.323572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.323810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.324075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] standard_threads = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.324338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.324617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.324795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.324975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_journal = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.325155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_json = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.325335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_stderr = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.325515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_syslog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.325749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.326008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.326249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.326511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.326752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.327003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.327243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.327493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.327731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.328005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.328258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.328512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.328749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.328997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.329236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.329487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.329738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.329978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.330235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.330490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.330738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.330976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.331233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.331497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.331753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.332032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.332274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.332506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.332739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.332978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.333229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.333480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.333712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.333944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.334193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.334441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.334682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.334933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.335167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.335404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.335643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.335906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.336142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.336377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.336610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.336851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.337102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.337335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.337576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.337809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.338048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.338288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.338540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.338775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.339014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.339254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.339493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.339732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.340559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.340750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.341002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.341240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.341481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.341715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.341978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.342217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.342450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.342696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.342932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.343207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.343447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.343698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.343970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.344213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.344461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.344701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.344952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.345193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.345433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.345679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.345917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.346157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.346395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.346632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.346869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.347134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.347375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.347617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.347878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.348126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.348365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.348602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.348840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.349073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.349305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.349548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.349786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.350017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.350093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 13:40:44.350222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 13:40:44.350300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.350421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] command line args: [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 13:40:44.350499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.350698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 13:40:44.350775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.350883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ================================================================================ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 13:40:44.350962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.351143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.351220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.351396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_dir = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.351479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.351626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.351701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.351771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] config_source = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.351908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.external = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.351987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] debug = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.352143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.352223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.352397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.352475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.352584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.352667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.352842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.352921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.password = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.353102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.353180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.token = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.353339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.353425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.353524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_config_append = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.353606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.353707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.353806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.353888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_dir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.354040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.354119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.354285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.354366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.354532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.354613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.354768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.354846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.355004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.355081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.355203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.355280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.355396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.355515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.356332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.356448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.356578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.356826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.356946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.357065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.357195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.357335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.357462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.357568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.357651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.357811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.357896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.358063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.358145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.358318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.358399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.358580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.358660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.358845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.359088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.359174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.359278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] publish_errors = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.359362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.359520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.359610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.359759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.359879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.359967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.360067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.360218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.360316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.360461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.360548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.360704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] standard_threads = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.360782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.360943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.361195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.361380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.361563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_journal = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.361796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_json = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_stderr = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.361991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] use_syslog = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.362194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 13:40:44.362417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.362917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.363877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.364960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.365894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.366885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.367896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.368854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369815 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.369985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.370934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.371912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.372967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.373917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.374882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.375844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.376973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.377994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.378959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.379962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.380902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.381876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.382983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.383060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.383231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.383584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.383748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.383850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.384984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.385944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.386944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.387962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.external = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.388921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.password = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] auth.token = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.389888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.390862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.391950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.392958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.393945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.394926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.395995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.396963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.397942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.398901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.399864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.400946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.401967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.402046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.402204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.402290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.402978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.403848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.404975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 13:40:44.405723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.405932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.406194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.406434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.406686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.406938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.407173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.407419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.407655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.408049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.408301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.408550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.408796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.409039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.409297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.409550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.409787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.410023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.410278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.410530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.410771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.411013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.411253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.411488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.411732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.412019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.412316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.412591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.412844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.413099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.413339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.413572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.413853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.414093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.414334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.414669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.414927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.415182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.415435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.415688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.415965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.416203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.416454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.416697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.416941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.417184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.417425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.417674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.417926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.418800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.419024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.419262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.419513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.419757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.420100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.420344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.420598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.420900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.421174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.421453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.421735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.422025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.422315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.422593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.422887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.423168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.423409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.423661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.424195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.424447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.424708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.424965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.425200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.425457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.425710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.425964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.426218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.426472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.426724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.426976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.427229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.427466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.427716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.427992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.428247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.428497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.428904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.429145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.429400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.429655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.429906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.430160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.430400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.430670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.430929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.431172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.431408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.431644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.431906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.432164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.432423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.432675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.432923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.433161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.433397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.433636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.434018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.434279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.434521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.434783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.435036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.435278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.435527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.435775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.436053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.436294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.436539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.436617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.436777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.436859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.437028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.437107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.437281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.437516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.437764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.438059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.438335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.438587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.438838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.439082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.439339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.439581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.439858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.440091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.440333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.440584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.440826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.441075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.441315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.441569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.441811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.442060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.442295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.442538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.442782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 13:40:44.442958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80166) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 13:40:44.445605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.468915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.469146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.469380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.474924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.475163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.479400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.497964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.498201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.530654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.538669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.582653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.582880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.587201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.587448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.649360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.649603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.649719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.649817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.649960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.650865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.674278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.674512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.674719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.674805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.674955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.675841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.676070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.687226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.687478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.700399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.700703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.700830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.700919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.701905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.702042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.702170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80166) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 13:40:44.719503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.719719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 13:40:44.720100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.720246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.720381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 13:40:44.720526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 13:40:44.720658 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.720783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.720913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 13:40:44.721183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.721913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.733539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.733815 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.734083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.734362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.735023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.735310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.735580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.735889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.742516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.744076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.745835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.746118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.746525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.746815 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.747096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.747364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.747570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.747751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.747906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.748888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.749013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.749328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.749614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.749895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.750179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.750458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.776278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.778066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.778211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.778367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.779854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.780146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.783707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.783976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.784264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.785473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.785763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.786050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.791614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.791923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.793488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.793782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.796474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.796880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.797169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.797456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.797739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.798018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.798279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.798408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.798557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.798736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.798964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.799254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.799535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.799824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.800109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.800382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.817695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.817975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.819523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.819805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.822673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.824592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.826441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.826738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.828468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.828769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.831884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.833807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.835501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.837494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.839391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.839694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.840566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.840857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.841280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.841568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.841843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.842148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.842442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.842730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.843017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.843318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.860958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.861240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.861508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.861774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.862447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.862735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.863006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.863272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.869809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.871895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.872199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.872476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.873625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.873911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.875554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.875854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.876142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.876425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.876700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.877000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.877285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.877581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.877869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.878151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.890566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.890847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.891127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.891407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.891689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.892002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.892303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.892431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.892612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.892899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.893199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.893488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.893771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.894053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.911030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.913336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.914358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.916660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.917684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.919395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.920217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.921067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.921350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.921630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.921855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.923479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.923767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.924089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.929070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.930693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.931801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.933445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.934671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.934965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.935253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.935531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.937259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.937554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.937842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.938135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.944709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.944989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.945378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.947336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.947628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.948048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.953364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.953644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.953926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.954211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.954486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.954765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.955058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.955327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.955604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.955908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80166) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 13:40:44.956183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.956940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.957023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.957215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.957489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.957769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.958057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.958333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.958615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.958890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.959163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.959434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.959708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80166) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 13:40:44.996737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e9489fa9c0 pid: 80167 (default app) May 24 13:40:44.999980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e9489fa9c0 pid: 80166 (default app) May 24 13:40:45.000182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e07ae-15da-4973-a7a4-c145f5af0969 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:45.000484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e07ae-15da-4973-a7a4-c145f5af0969 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:45.000767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e07ae-15da-4973-a7a4-c145f5af0969 None None] PATH_INFO: `/v3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:45.005030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1/1] 10.222.0.24 () {50 vars in 791 bytes} [Tue May 24 13:40:44 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:40:52.932421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:52.932945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:52.932945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:52.938165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1/2] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:40:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:40:52.943923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:52.944340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:52.944496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:53.012287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: INFO dbcounter [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] Registered counter for database keystone May 24 13:40:53.033692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG oslo_db.sqlalchemy.engines [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 13:40:53.039713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writer thread running {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 13:40:53.215888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 13:40:53.216247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 13:40:53.271991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:40:53.273603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:53.397111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 2/3] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:40:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 13:40:53.554328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:53.622215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: INFO dbcounter [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] Registered counter for database keystone May 24 13:40:53.642615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG oslo_db.sqlalchemy.engines [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80166) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 13:40:53.648687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writer thread running {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 13:40:53.769114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6214c91-eca3-4220-8150-f47bac0410c3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:53.773205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:53.773872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:53.774160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:53.774441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:53.782634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:53.833808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 13:40:53.834151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d720d7-201f-4c62-9762-af8272f9e4ca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:53.844044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 2/4] 10.222.0.24 () {58 vars in 1324 bytes} [Tue May 24 13:40:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:40:53.850123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:53.864735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f5699-78a9-4f51-869f-52b3d631f521 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:53.869127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:53.870248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:53.870543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:53.870832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] PATH_INFO: `/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:53.872917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] RBAC: Authorizing `identity:get_project(project_id=4d5cb275b4fb4682b096f6cd5768d3fe)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:54.032066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 13:40:54.032664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70b8789-d90e-4b34-97c1-5f272713925f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:54.037084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 3/5] 10.222.0.24 () {56 vars in 1207 bytes} [Tue May 24 13:40:53 2022] GET /identity/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:40:56.209846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aae1f-0994-4201-a526-79344edd73fd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:56.210423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aae1f-0994-4201-a526-79344edd73fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:56.210423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37aae1f-0994-4201-a526-79344edd73fd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:56.214571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 3/6] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:40:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:40:56.220749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:56.221163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:56.221330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:56.240498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:40:56.242009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:56.249806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 4/7] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:40:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 13:40:56.271774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aae1f-0994-4201-a526-79344edd73fd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:56.284148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37aae1f-0994-4201-a526-79344edd73fd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:56.288338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:56.288974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:56.289267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:56.289554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:56.298104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:56.299895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae7458-b58f-49be-b86a-42ecf9c4817a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:56.308854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 4/8] 10.222.0.24 () {58 vars in 1324 bytes} [Tue May 24 13:40:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:40:56.313993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:56.322770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32909861-f076-42b4-88cd-8bf339f68fda None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:56.327275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:56.328171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:56.328465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:56.328742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:56.330120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:56.331958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12755984-e707-4ab1-9ae8-9528eab3c13e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:56.333498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 5/9] 10.222.0.24 () {56 vars in 1198 bytes} [Tue May 24 13:40:56 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:40:58.596042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea622-f6ad-4a08-9df2-dd0a29f268f9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.596574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea622-f6ad-4a08-9df2-dd0a29f268f9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.596662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ea622-f6ad-4a08-9df2-dd0a29f268f9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:58.597950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 5/10] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:40:58.603215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.603629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.603788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:58.620169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:40:58.621890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:58.632749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2d16a-391b-4d82-b3f2-a22d50888f35 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.633110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2d16a-391b-4d82-b3f2-a22d50888f35 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.633705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2d16a-391b-4d82-b3f2-a22d50888f35 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:58.635108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 6/11] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:40:58.640948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.641241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.641525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:58.681258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80166) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 13:40:58.681689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG passlib.handlers.bcrypt [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80166) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 13:40:58.701881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 6/12] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:40:58.724508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:58.729655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:40:58.731928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:40:58.748406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:58.830299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aeaf7-fd9c-4ed9-a883-855eddd2c47a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:58.832807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 7/13] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:40:58.854316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:58.863456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:58.864305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.864597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.864883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:58.877037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:40:58.888590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:40:58.889589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:40:58.890546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 7/14] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:40:58.895452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:58.956648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02254b2e-61f1-4d38-886a-d48dbb0f3171 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:58.959668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:58.975347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:58.994093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db50c31-da26-45f6-9841-a56b339356b0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:58.996144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:58.997087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:58.997361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:58.997648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.028053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.028718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.029011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.029293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.030053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:40:59.030367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.031770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.043216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 8/15] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:40:59.048759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:59.064152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.065682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.148316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cdca0-9e08-45d4-b47e-15d1be11f170 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:59.164452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 8/16] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:40:59.170136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:59.172845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:59.181829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.182704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.182994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.183870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.185260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.186725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.188360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 9/17] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:40:59 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:40:59.193504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:59.208242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:59.271388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26579cdd-88b4-4813-841b-cfdcd9c92928 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:59.294709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cec0a-00d9-49b9-8901-8c23ade5b43b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:59.304739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.305527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.305812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.306084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.323772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.325421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.327392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.328221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.328509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.328790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.329981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.331440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.333271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 10/18] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:40:59.338615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:40:59.346842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:40:59.347631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:40:59.348616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:40:59.348616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:40:59.348616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:40:59.348616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:40:59.348616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 9/19] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:40:59.353662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:40:59.409360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:59.420365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:40:59.439053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd91fa-269a-4669-a89d-1a458b18f972 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:59.457308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f696b6-7421-4d53-8bab-3101e4b878e8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:40:59.474307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.474960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.475244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.475521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.476382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.domain.id=default, user.id=a064d7a4fdb343efa665d8a825cdfba7) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:40:59.476703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.478153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c8485-9576-4f38-9b34-ebba8ea1fb4c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.485123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 11/20] 10.222.0.24 () {56 vars in 1322 bytes} [Tue May 24 13:40:59 2022] GET /identity/v3/role_assignments?user.id=a064d7a4fdb343efa665d8a825cdfba7&scope.domain.id=default&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:40:59.495534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:40:59.496316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:40:59.496603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:40:59.496883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:40:59.497735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:40:59.497971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:40:59.501440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ae6e3-5038-478a-9de7-0e6c96b1767d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:40:59.524503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 10/21] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:01.971747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f90d8-fc62-4288-9ee7-e776387ad323 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:01.972252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f90d8-fc62-4288-9ee7-e776387ad323 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:01.972252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f90d8-fc62-4288-9ee7-e776387ad323 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:01.973926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 12/22] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:01.979638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:01.980590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:01.980788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.122220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:02.123428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:02.215729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 11/23] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:02.258123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f90d8-fc62-4288-9ee7-e776387ad323 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:02.394512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f90d8-fc62-4288-9ee7-e776387ad323 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:02.451899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:02.452727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.453023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.453312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.468577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c37b9c23-563f-46db-9051-ee5da707515c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:02.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:02.477210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:02.477945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 13/24] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:02.484148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:02.552845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:02.635162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa7fc8-2736-4132-910b-891b315f618d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:02.645458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7286b-869c-42b8-9e1f-0cfb37587029 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.645756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7286b-869c-42b8-9e1f-0cfb37587029 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.646043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7286b-869c-42b8-9e1f-0cfb37587029 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.647525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 14/25] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:02.653158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a138-bcb9-46de-b535-353ea49399f3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.653455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a138-bcb9-46de-b535-353ea49399f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.653746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03a138-bcb9-46de-b535-353ea49399f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.672222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:02.692135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fa03a138-bcb9-46de-b535-353ea49399f3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:02.693366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fa03a138-bcb9-46de-b535-353ea49399f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:02.723668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:02.724382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.724674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.724958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.725734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:02.726047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:02.727455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:02.752135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 12/26] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:02.765118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:02.830832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 15/27] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:02.840354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c604f-2fac-403e-b702-c87d11858c26 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.840826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c604f-2fac-403e-b702-c87d11858c26 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.841421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770c604f-2fac-403e-b702-c87d11858c26 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.842929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 16/28] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:02.848038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d9ae-66f9-4adf-8cc3-07513ec004e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.848484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d9ae-66f9-4adf-8cc3-07513ec004e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.848904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9d9ae-66f9-4adf-8cc3-07513ec004e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.888795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d138d38-7443-44fc-9056-91b7cf5f49b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:02.890052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f4c9d9ae-66f9-4adf-8cc3-07513ec004e7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:02.894576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f4c9d9ae-66f9-4adf-8cc3-07513ec004e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:02.906130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:02.944575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:02.945705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.945987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.946274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.948670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:02.951524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:02.954836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 13/29] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:02 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:02.955657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:02.983151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 17/30] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:02.984586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5368-9ecf-4652-a001-a894b01a71ff None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:02.985411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5368-9ecf-4652-a001-a894b01a71ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:02.985714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c5368-9ecf-4652-a001-a894b01a71ff None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:02.987664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 18/31] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:02.993029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5368-9ecf-4652-a001-a894b01a71ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.067189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:03.082501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee82fc3-2ec6-4b93-8327-14450483174d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.110479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c5368-9ecf-4652-a001-a894b01a71ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.120510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.122131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.122628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.123109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.134555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.136479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.148818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.149622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.149913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.150197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.151493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.153666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575663b-2e6f-44f2-be2a-896cd77cffb1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.155452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 19/32] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:03.156687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.156984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.157271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:41:03.157532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:03.158267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:03.159062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:41:03.159062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:41:03.159062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:41:03.159062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:03.161060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 14/33] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:03.162076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.183713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:03.184993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:03.194793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:03.272628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 20/34] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:03.273468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.280966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0217e-bb44-4c6f-bf2e-4b9e50055a31 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.322138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.322846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.323136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.323418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.324400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.326040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.343262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:03.383326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33fa7d8-15e0-48c1-81bc-4285f54a2dc6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.423224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.424137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.424429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.424714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] PATH_INFO: `/v3/domains/default/users/a064d7a4fdb343efa665d8a825cdfba7/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.426722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a064d7a4fdb343efa665d8a825cdfba7, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.428452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaafab2-5cb6-4d03-b1d3-b599937b3a14 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.455849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 15/35] 10.222.0.24 () {60 vars in 1131 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/roles => generated 219 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:03.455849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.556704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 21/36] 10.222.0.24 () {58 vars in 1361 bytes} [Tue May 24 13:41:03 2022] PUT /identity/v3/domains/default/users/a064d7a4fdb343efa665d8a825cdfba7/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:03.559098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba3eb-ab4b-4ebd-983b-13a4f86ff166 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.559409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba3eb-ab4b-4ebd-983b-13a4f86ff166 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.559702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42ba3eb-ab4b-4ebd-983b-13a4f86ff166 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.564271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 22/37] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:03.565130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba3eb-ab4b-4ebd-983b-13a4f86ff166 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.568997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:03.604519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8f3ed-e8f2-4817-8612-86645cfe148c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.694663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.695423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.695723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.696040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.696825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:03.697147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.700617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063bb4d-5968-4459-8dc0-3c512bca08d6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.725784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 16/38] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:41:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:03.726995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.727296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.727586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.732627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42ba3eb-ab4b-4ebd-983b-13a4f86ff166 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.758352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:03.760671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:03.769325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:03.785320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.786758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.787064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.787355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.788175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.789755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:03.812379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 23/39] 10.222.0.24 () {60 vars in 1131 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/roles => generated 225 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:03.813179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.877205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 17/40] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:03.919215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:03.926938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadf379-5a4a-4f88-bf40-028e2895a8b3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:03.966416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:03.967183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:03.967486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:03.967777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:03.970644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:03.970644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef3c89e-abff-46e0-a18a-582339e7a045 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:04.007217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 24/41] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/projects => generated 301 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:04.052892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89ff9-965a-4691-b4a0-a5200e43eefa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:04.053494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89ff9-965a-4691-b4a0-a5200e43eefa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:04.053791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a89ff9-965a-4691-b4a0-a5200e43eefa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:04.055956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 25/42] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:04.063176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:04.063468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:04.064860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:04.077092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:04.111470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1189e4-adb7-4205-8af3-273fdf57e539 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:04.117167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:04.124737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:04.166737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:04.168712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:04.169359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:04.169870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:04.170973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:04.172323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:04.172455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:04.198477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 18/43] 10.222.0.24 () {60 vars in 1131 bytes} [Tue May 24 13:41:03 2022] POST /identity/v3/roles => generated 223 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:04.237439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 26/44] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:04.264283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:04.387028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f486d-8fa9-4556-bec0-15fcf5d44f60 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:04.425696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:04.426430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:04.426728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:04.427013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:04.428290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:04.429742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b64f7c-ec88-4209-8ea1-a52551ae5e7f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:04.431364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 19/45] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:04.437721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:04.545916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff5e53-677b-4e14-a0e6-f7c78b9639da None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:04.579761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:04.580463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:04.580754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:04.581035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:04.581759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:04.583218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dc7b4-dfa2-4db8-88fd-cdcf2ebbc28b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:04.606878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 27/46] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:41:04 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:05.969342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc854a-2f9d-448e-9d6a-13cd1e887c2c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:05.969826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc854a-2f9d-448e-9d6a-13cd1e887c2c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:05.969826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc854a-2f9d-448e-9d6a-13cd1e887c2c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:05.971195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 20/47] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:05.976473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:05.976885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:05.977058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:05.993534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:05.994720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:06.005721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:06.061547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 28/48] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:06.084790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc854a-2f9d-448e-9d6a-13cd1e887c2c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:06.094113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:06.187139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc854a-2f9d-448e-9d6a-13cd1e887c2c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:06.218938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:06.219786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.220056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.220338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.232559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:06.241300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:06.242276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:06.243092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 21/49] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:06.248349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:06.343987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6c08c-8e67-48e1-ab9b-0872408fa7e6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:06.377780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:06.378468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.378761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.379047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.379834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:06.380155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:06.381569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:06.390090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 29/50] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:06.395489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:06.459699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:06.493909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ec63-a5b1-46f5-8cda-e096bdffd9a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:06.530736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:06.531614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.531945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.532262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.533485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:06.534953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:06.536640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 22/51] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:06.541952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:06.553887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:06.642535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a466f0-7704-47a7-98f6-ae3383deb7e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:06.678420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:06.679226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.679524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.679835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.681025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:06.682496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12763bd-0563-48b0-88e1-9c250780e589 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:06.684406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 30/52] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:06.689967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:06.788342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055e75f-a2d9-438d-b51a-b42294131d1c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:06.821542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:06.822214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.822508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.822800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.823695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.domain.id=default, user.id=a064d7a4fdb343efa665d8a825cdfba7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:06.824212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:06.825666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:06.825885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df082-7c1b-424f-b173-ae2758d2dfde None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.826186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df082-7c1b-424f-b173-ae2758d2dfde None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.826476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4df082-7c1b-424f-b173-ae2758d2dfde None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.827863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 31/53] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:06.828105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:06.833191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:06.833493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:06.833777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:06.843885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 23/54] 10.222.0.24 () {56 vars in 1322 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/role_assignments?user.id=a064d7a4fdb343efa665d8a825cdfba7&scope.domain.id=default&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:06.852009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:06.853276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:06.930522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 32/55] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:06.956482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:07.059908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682901a-4c78-4b2e-bddf-ada4ef9bf451 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:07.096715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:07.097517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:07.097813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:07.098100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:07.099267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:07.100757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3a6450-e121-4cdf-bda2-fc99d1c3138a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:07.102371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 24/56] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:07.108079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:07.116165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:07.205556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed8044c-494b-4bdb-8c50-5e46ed08e356 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:07.239426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:07.240109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:07.240398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:07.240678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:07.241389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:07.242831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d11158-f560-44c9-87b9-6b6254ac17d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:07.274457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 33/57] 10.222.0.24 () {60 vars in 1140 bytes} [Tue May 24 13:41:07 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:09.463280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a281-750e-4ee3-9bad-ac1a628f03a1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:09.463850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a281-750e-4ee3-9bad-ac1a628f03a1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:09.463850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5a281-750e-4ee3-9bad-ac1a628f03a1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:09.466165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 25/58] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:09.471228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:09.471623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:09.471807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:09.488300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:09.489492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:09.542650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:09.556725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 34/59] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:09.581312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5a281-750e-4ee3-9bad-ac1a628f03a1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:09.645323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:09.679280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5a281-750e-4ee3-9bad-ac1a628f03a1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:09.712062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:09.712849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:09.713141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:09.713469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:09.714581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:09.716604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bbd12-737f-4583-9423-e5b3adaf47ed None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:09.718202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 26/60] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:09.723670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:09.819980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cf1397-e0f6-4781-95dd-587df75a0fef None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:09.852573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:09.853244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:09.853533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:09.853817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:09.854542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:09.856019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d16e0-53a7-4f49-8f5a-33e90e548d66 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:09.880470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 35/61] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:09 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:13.013222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fff29-aec7-4ea0-8f26-ab4e1cd3bbcf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.014367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fff29-aec7-4ea0-8f26-ab4e1cd3bbcf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.014863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57fff29-aec7-4ea0-8f26-ab4e1cd3bbcf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.016550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 27/62] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:13.022015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.022850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.023345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.048915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:13.050128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:13.102928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:13.141310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 36/63] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:13.164007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fff29-aec7-4ea0-8f26-ab4e1cd3bbcf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:13.324659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57fff29-aec7-4ea0-8f26-ab4e1cd3bbcf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:13.371357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:13.372244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.372537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.372824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.374937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:13.388391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-474cf703-26d8-4ea9-bca9-4d63f1d873f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:13.402589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:13.404092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:13.404876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 28/64] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:13.410221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:13.563547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3811ab-213d-46e3-956f-1fa21d7a6d2b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:13.625611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:13.626234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.626526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.626808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.627583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:13.627926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:13.629343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:13.644997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a33c2-fb8b-4876-9fdd-0808f3f903e7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.645301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a33c2-fb8b-4876-9fdd-0808f3f903e7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.645572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a33c2-fb8b-4876-9fdd-0808f3f903e7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.647068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 29/65] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:13.647524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 37/66] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:13.652598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6642-3990-457f-b541-2b68dcd9a075 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.652895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6642-3990-457f-b541-2b68dcd9a075 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.652993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:13.653217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d6642-3990-457f-b541-2b68dcd9a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.671082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-880d6642-3990-457f-b541-2b68dcd9a075 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:13.672550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-880d6642-3990-457f-b541-2b68dcd9a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:13.692300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:13.761515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 30/67] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:13.763590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c731d5-777d-41ef-b458-0e18b1f85fd1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.763989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c731d5-777d-41ef-b458-0e18b1f85fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.764466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c731d5-777d-41ef-b458-0e18b1f85fd1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.765904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 31/68] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:13.769026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a88e-7346-4107-b316-5e92f768347a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.769527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a88e-7346-4107-b316-5e92f768347a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.770038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8a88e-7346-4107-b316-5e92f768347a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.771448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 32/69] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:13.775533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.775533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.775533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.790202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:13.791394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:13.799112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454ee56-ba7d-4706-9596-947a7d7d976b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:13.836078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:13.836961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.837255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.837534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.851231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:13.852748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cf64e-041c-494b-9454-f985334ad3fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:13.854426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 38/70] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:13 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:13.855757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:13.856205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:13.857163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:13.873422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:13.874611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:13.875213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 33/71] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:13.876004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:13.942892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:13.957587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 39/72] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:13.958451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:13.986067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:13.993361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eaf88-0b60-4aaf-a1ed-49e8312de013 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.030162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.030970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.031260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.031547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.041410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:14.050085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:14.051095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 34/73] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:14.052049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.064193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57742456-c931-4e60-a3bf-1cb6c00a8575 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.098414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.099071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.099365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.099646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.151152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebb210-5ab9-4f6e-9a53-edaa16ef123a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.165292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.165492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.166971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.186563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.187355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.187646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.187956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.190619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.198268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-eba35fc9-278c-45a6-9a01-0f297460312f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:14.205976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:14.206929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:14.207652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 35/74] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:14.209051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87001b-7aa3-440d-b836-f75da3150932 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.209051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87001b-7aa3-440d-b836-f75da3150932 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.209051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87001b-7aa3-440d-b836-f75da3150932 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.210194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 36/75] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:14.210891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87001b-7aa3-440d-b836-f75da3150932 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.269067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 40/76] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:14.269930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.312281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87001b-7aa3-440d-b836-f75da3150932 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.345714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.346506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.346801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.347086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.348306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.349781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.351421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 37/77] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:14.352205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.365558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c3f5-4648-40a1-bc45-fec2fb591173 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.388729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.398999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.400040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.400722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.401014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.401451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.402099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:14.402418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.404445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc75db8-ab0a-4e88-88ac-8145bfdfc7b7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.412423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 41/78] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:14.413627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.413921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.414205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.429818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:14.431002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:14.462286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e069c-d03e-4cf7-adc0-33c515117b04 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.497096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.497317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 42/79] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:14.498116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.498229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.498637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.499059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.500023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:14.500546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.502238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.520453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 38/80] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:14.521492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.600632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4e00-4edf-4abe-a0de-b2975be196f9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.616991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.618919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588c9da-1bb5-4d6e-825b-2dd8546a2d53 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.621810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.635584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.636498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.636791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.637075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.638929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.640440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.642143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 43/81] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:14.642963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.655845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.656465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.656771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.657060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.657803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.659222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.699847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 39/82] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:14 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:14.699847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.769157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1bd2c-939f-4db8-92de-31fcaeb3a2c7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.799089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5208a-aa78-4920-99d9-c05066681929 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.812220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.813130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.813430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.813719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.814998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:14.816585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:14.818244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 44/83] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:14.823048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.847099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.848522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.849105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.849672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.862796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:14.870601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:14.871434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:14.872846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 40/84] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:14.873466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:14.884787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.899034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:14.946544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa96ef24-c5c1-454e-ab02-eff0ef28404a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:14.996394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:14.997874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:14.998107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:14.998382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:14.999736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.001250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.002922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 45/85] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:15.003711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.015570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15786d69-6832-49d0-926f-02c8bc9ea125 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.057521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.058186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.058481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.058765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.060197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:15.060768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.062256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.069495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 41/86] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:14 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=b82b050f41064700b9690d91633b2c52 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:15.070565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.123902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79259086-6bd0-4558-ae81-c4af7fcb3c16 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.152263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:15.163316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.164057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.164351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.164640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.241718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:15.244654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.246165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.301980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c1afb9-ef50-4b57-b36d-e1a7d336208d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.367629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 46/87] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:15.368772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.375508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.376510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.377096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.377592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.378610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:15.379122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.380852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.393848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 42/88] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:15.395017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.395281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:15.498980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4e991-2d73-4d6d-9fd0-ed4b9deeeaa9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.569865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec7445-6221-430f-89b3-9428473c47f5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.571881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.572549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.572840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.573124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.637849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.638732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.639029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.639319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.646728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.648246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.650029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 43/89] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:15.650815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.706904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:15.728624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.730457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.768607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:15.799971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdfd91-90dc-4072-85da-960e62112397 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:15.864537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:15.865429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:15.865719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:15.866001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:15.867319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:15.868850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:15.871742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 44/90] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:15.873369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.874283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 47/91] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:15.879897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:15.989339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:16.006349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97730e66-1908-4f85-98c1-9b365883dbc8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.030485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:16.038013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f9194-7116-4e6d-8b4d-a2f479178804 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.048014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.048898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.049189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.049475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.051259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.052806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.054687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 48/92] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:16.055288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:16.129826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.130572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.130837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.131125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.132008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:16.132335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.136812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.150818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 45/93] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:15 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:16.151569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:16.173618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44d0f4-ae80-4ad5-8dfd-78d78c67ecf3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.219407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.220516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.220979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.221418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.298490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:16.316725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.318471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.402112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3930-3993-4d32-90a9-dc964eb33885 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.443469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 49/94] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:16.450378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:16.462562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.463438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.463921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.464342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.465319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=a064d7a4fdb343efa665d8a825cdfba7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:16.466314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.468212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.474092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 46/95] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:16 2022] GET /identity/v3/role_assignments?user.id=a064d7a4fdb343efa665d8a825cdfba7&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:16.564030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:16.578232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07628b-4e62-4e0f-ba09-1252b0254333 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.617319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.618189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.618476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.618755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] PATH_INFO: `/v3/projects/0cfa61b3cc4f4aa48a071128544d246e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.619984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] RBAC: Authorizing `identity:get_project(project_id=0cfa61b3cc4f4aa48a071128544d246e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.621566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e49334-3a06-423e-93d0-fd143adafef6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.624406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 50/96] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:16 2022] GET /identity/v3/projects/0cfa61b3cc4f4aa48a071128544d246e => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:16.630192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:16.645505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:16.750884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449d05d-d447-4e82-89dc-ebe532da6b43 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:16.789710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:16.790375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:16.790664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:16.790947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:16.791796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=0cfa61b3cc4f4aa48a071128544d246e, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:16.792135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:16.794265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae16f23-d3f8-4d98-b9b6-ee0c990648cc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:16.801057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 47/97] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:16 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=0cfa61b3cc4f4aa48a071128544d246e&role.id=b82b050f41064700b9690d91633b2c52 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:17.843677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed360d-51e2-4128-812e-b5cc9ffd8902 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:17.844696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed360d-51e2-4128-812e-b5cc9ffd8902 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:17.845193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed360d-51e2-4128-812e-b5cc9ffd8902 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:17.846829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 51/98] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:17.852595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:17.853444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:17.853940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:17.877353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:17.878936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:17.971981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 48/99] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:17.999863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed360d-51e2-4128-812e-b5cc9ffd8902 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.140619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed360d-51e2-4128-812e-b5cc9ffd8902 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:18.186730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:18.187945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.188450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.188956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.196562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:18.198275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d92d7-cf6f-4814-85e7-bb3f447c3d9d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:18.200891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 52/100] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:18.206636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.210111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:18.266030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070951e-9cdf-4638-b5ab-eca37d17b98e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.269722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070951e-9cdf-4638-b5ab-eca37d17b98e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.270272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070951e-9cdf-4638-b5ab-eca37d17b98e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.271948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 53/101] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:18.273305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b896b4-3b34-4c56-97f0-5827ac58debe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.273828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b896b4-3b34-4c56-97f0-5827ac58debe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.274315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b896b4-3b34-4c56-97f0-5827ac58debe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.275981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 54/102] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:18.281417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.281965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.282457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.316824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:18.318347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:18.336004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8004c7-9413-4138-9e4e-ce36d003637e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:18.373830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:18.378404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:18.379096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.379385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.379675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.380439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:18.381892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce12b2-4ecb-4ace-99f9-a561af00f39d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:18.437545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 49/103] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:41:18 2022] POST /identity/v3/projects => generated 302 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:18.439110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.439598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.440103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.463850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 55/104] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:18.470895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:18.472336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:18.490659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.562032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:18.602112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 50/105] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:18.640969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.681157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500f6d3-4b13-4c2e-b1f9-eb2ae3db9899 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:18.763965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:18.765130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.765610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.766078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.796990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:18.805254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:18.806212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:18.807026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:18.810445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 56/106] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:18.816324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.817999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a73fb-5b25-4139-aed3-e686878151a2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:18.849599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:18.906713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:18.909197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:18.909499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:18.909789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:18.928932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:18.942368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:18.943272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:18.944046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 51/107] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:18.956853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:18.994693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3a635-2347-4862-9f4d-36e231c64299 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.044602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.045905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.046787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:19.047619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.048284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.049451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:19.050732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.051839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.076365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 57/108] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:19.082182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:19.176522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bcdad-91cc-4dc2-8886-966127b8e2fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.243650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:19.273481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.274521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.275029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.275514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.276513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:19.277020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.279679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.291219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d331a-a02d-4cda-8576-917c4b611d20 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.302144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 52/109] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:19.313060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:19.356841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.358228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.358734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.359210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] PATH_INFO: `/v3/users/f11416539ba24a47916d4766fdffd7ce` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.386639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] RBAC: Authorizing `identity:get_user(user_id=f11416539ba24a47916d4766fdffd7ce)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.389143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.391133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 58/110] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:19 2022] GET /identity/v3/users/f11416539ba24a47916d4766fdffd7ce => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:19.397514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:19.585572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a866b-64d5-46ef-b283-e5ff3c4d25e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.612185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:19.638660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a70ca-2877-45d9-92ba-991454281734 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.692785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.693704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.694000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.694287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.701431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.703204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f0ddb-1b57-4b97-961a-0c8d52a5fedf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.705196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 53/111] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:19 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:19.706636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7253-08d1-40b8-a3a2-0318c238c7ac None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.707091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7253-08d1-40b8-a3a2-0318c238c7ac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.707523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7253-08d1-40b8-a3a2-0318c238c7ac None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.717572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 54/112] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:19 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:19.718573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7253-08d1-40b8-a3a2-0318c238c7ac None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:19.740599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.741280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.741572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.742777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.751582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:19.862558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.864729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85e1f1e-475e-4810-bba9-a3c40754aa36 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.952911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7253-08d1-40b8-a3a2-0318c238c7ac None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:19.987707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 59/113] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:19.989310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ba79-8d57-4a6d-9ba9-7ecd5535eb45 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.989612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ba79-8d57-4a6d-9ba9-7ecd5535eb45 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.989924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee1ba79-8d57-4a6d-9ba9-7ecd5535eb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.992226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:19.993427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:19.993976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:19.994403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:19.995863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:19.997804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73782-12b0-4a4f-88e1-77ad47f37de1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:19.999712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 55/114] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:19 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:20.001270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84db4f-e979-4bc8-9e3f-d9f72f30828b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.001768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84db4f-e979-4bc8-9e3f-d9f72f30828b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.002230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84db4f-e979-4bc8-9e3f-d9f72f30828b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.003649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 56/115] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:20.004638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84db4f-e979-4bc8-9e3f-d9f72f30828b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.010305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6ee1ba79-8d57-4a6d-9ba9-7ecd5535eb45 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:20.011606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6ee1ba79-8d57-4a6d-9ba9-7ecd5535eb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:20.041256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.058801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.091525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 60/116] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:20.093594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04cfd-45e6-4229-b6e0-381b7d273726 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.094112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04cfd-45e6-4229-b6e0-381b7d273726 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.094544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac04cfd-45e6-4229-b6e0-381b7d273726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.119002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bac04cfd-45e6-4229-b6e0-381b7d273726 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:20.120703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bac04cfd-45e6-4229-b6e0-381b7d273726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:20.129213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84db4f-e979-4bc8-9e3f-d9f72f30828b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.164542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.165840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.166289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.167332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.169147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:20.170970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:20.177766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 57/117] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:20.177766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 61/118] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4387-f2e8-4008-b736-119a4d81e5a3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4387-f2e8-4008-b736-119a4d81e5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4387-f2e8-4008-b736-119a4d81e5a3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 62/119] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:20.203860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4387-f2e8-4008-b736-119a4d81e5a3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.299368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa4b96-4b72-4bf7-98fd-bce9db082407 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.310865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.316586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.320485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4387-f2e8-4008-b736-119a4d81e5a3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.338936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.339852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.340143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.340966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/bf9b7c7fda6947388c02a47587519de9/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.343219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, user_id=bf9b7c7fda6947388c02a47587519de9, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:20.344704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:20.359375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.372524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 58/120] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:20 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/bf9b7c7fda6947388c02a47587519de9/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:20.372801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.372801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.372801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.373207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.386094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0328a0fc-39bd-42c3-ab7c-cd532cf521cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:20.395548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:20.396548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:20.397282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:20.402321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 63/121] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:20.403915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.404223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.404568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.423959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:20.425193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:20.520713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7ad66-7d94-4201-a201-acf09c1e00fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.527841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 64/122] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:20.528410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.577306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.577996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.578291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.578578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.579510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=f11416539ba24a47916d4766fdffd7ce) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:20.579858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:20.581336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:20.591047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 59/123] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/role_assignments?user.id=f11416539ba24a47916d4766fdffd7ce&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:20.591495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.645667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.649546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.664278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90c09d-7790-4fb2-a008-42f0f83c6105 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.709993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.711236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.711759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.712296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.724978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:20.733616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1bfea-725a-4ba6-8797-ebb8245876c4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:20.734415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:20.735222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:20.736353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:20.738504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 65/124] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:20.739532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.780353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.781048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.781345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.781635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.782407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:20.782728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:20.784155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:20.799888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 60/125] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:20.799888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.899201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9d4b2-0e0c-4c84-9739-ebf3e042a403 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.919235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:20.924477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5af308-0403-4ac5-b6b6-21906d4aec9b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:20.946317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.947552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.948089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.948590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.967068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:20.967794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:20.968080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:20.968425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:20.971442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:20.971442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:20.971442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:20.979848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 61/126] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:20.980639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:20.980901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:20.989340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:20.990437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:20.991483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:20.998776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 66/127] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:20.999255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.004814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.098831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261172-2ecb-40fa-9b85-854b71a0140f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.155873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.156697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.156992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.157279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.159707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.161202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.163020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 62/128] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.163618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.204773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.224558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4f5f7-166f-437f-9080-e2143c23509c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.302035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.302934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.303227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.303511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.304770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.306249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.307910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 67/129] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:20 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.308690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.362603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f06213-7d29-4d7d-9aa5-e90e1c471ca6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.393533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.437979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.438968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.439535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.440744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.442290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:21.442903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.445695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.450457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6fa69-8021-47d8-bcd0-cbc6d77ddf18 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.455887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 63/130] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.456909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.494882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.495764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.496086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.496373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.498004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.500231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b3292-01f3-4b0e-a571-b7609977ffd0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.504536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 68/131] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.504536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750211-a899-415c-9420-6c6c08f5578e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.504536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750211-a899-415c-9420-6c6c08f5578e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.504536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11750211-a899-415c-9420-6c6c08f5578e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.510931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 69/132] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:21.512466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11750211-a899-415c-9420-6c6c08f5578e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.568064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.580024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a337ebb-0344-460d-96fc-f289cfd5577d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.619663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.620883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.621258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.622123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] PATH_INFO: `/v3/projects/0cfa61b3cc4f4aa48a071128544d246e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.623442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] RBAC: Authorizing `identity:get_project(project_id=0cfa61b3cc4f4aa48a071128544d246e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.625016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.626809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 64/133] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/projects/0cfa61b3cc4f4aa48a071128544d246e => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.627614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.646291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11750211-a899-415c-9420-6c6c08f5578e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.694702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.695807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.696535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.696965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.699272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.701063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862ed6c-6281-4540-b4e5-26176c613581 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.705289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 70/134] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.706868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.707294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.707711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.730398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:21.732709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:21.748144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.748691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2707263-c1e0-42a5-9660-1c87a6f6fe01 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.785197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.786103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.786383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.786668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/bf9b7c7fda6947388c02a47587519de9/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.789001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, user_id=bf9b7c7fda6947388c02a47587519de9, role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.790444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.803352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 71/135] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:21.804817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.812021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 65/136] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:21 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/bf9b7c7fda6947388c02a47587519de9/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:21.813104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:21.848543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:21.952054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63371c-7989-4b0c-acc9-c8c9f113dc1f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.961361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840dc89-c733-438d-ab5c-a028b31fad20 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:21.992040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:21.992943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:21.993233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:21.993517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:21.994929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:21.996535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:21.999111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 66/137] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:21.999811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:22.037262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:22.038170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:22.038459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:22.038744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] PATH_INFO: `/v3/projects/0cfa61b3cc4f4aa48a071128544d246e/users/bf9b7c7fda6947388c02a47587519de9/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:22.045588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] RBAC: Authorizing `identity:create_grant(project_id=0cfa61b3cc4f4aa48a071128544d246e, user_id=bf9b7c7fda6947388c02a47587519de9, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:22.047099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:22.071612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:22.082273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 72/138] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:21 2022] PUT /identity/v3/projects/0cfa61b3cc4f4aa48a071128544d246e/users/bf9b7c7fda6947388c02a47587519de9/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:22.083114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:22.136641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:22.180053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e6257-fcbe-4a86-8f2f-33219e142324 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:22.245097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce64769-213a-4416-9311-3b4cae8ebcfb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:22.247758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:22.248669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:22.248968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:22.249253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:22.250549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:22.252056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:22.255986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 67/139] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:22.265874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:22.282437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:22.283455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:22.283750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:22.284062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/a064d7a4fdb343efa665d8a825cdfba7/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:22.286082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, user_id=a064d7a4fdb343efa665d8a825cdfba7, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:22.287660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a0df4-73e8-424a-88f0-3827d34f1f3f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:22.312057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 73/140] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:22 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/a064d7a4fdb343efa665d8a825cdfba7/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:22.418378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07520e55-157d-4ac3-952e-6619ded2c22d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:22.457133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:22.457833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:22.458125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:22.458410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:22.459219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:22.460848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a676d95-e844-4e70-af99-725fec1f9e67 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:22.487675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 68/141] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:22 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:23.331680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d658506-b53b-4f07-86c7-e368a9dc3461 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.332208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d658506-b53b-4f07-86c7-e368a9dc3461 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.332283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d658506-b53b-4f07-86c7-e368a9dc3461 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.333805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 74/142] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:23.339330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.339763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.339970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.362150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:23.363348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:23.366438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:23.369570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243619be-af72-4f14-9129-a380e7c356fe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.369870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243619be-af72-4f14-9129-a380e7c356fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.370155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243619be-af72-4f14-9129-a380e7c356fe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.371578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 75/143] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:23.376846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.377186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.377477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.399164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:23.400415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:23.483369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:23.488297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 69/144] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:23.505593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 76/145] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:23.528671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:23.530187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:23.672976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e38372-e777-49be-9c8e-d3d80944c1fe None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:23.688235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8c7de-a6d5-4d5e-90fd-22f6018eab06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:23.728449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:23.729276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.729566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.729666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:23.729863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.730360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.730640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.731441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.742554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:23.752232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:23.753097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:23.753837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 77/146] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:23.759886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:23.761542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:23.768502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:23.776619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:23.777596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:23.778916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 70/147] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:23.804879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:23.836251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:23.881277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c3c61b-ce26-466b-a1f9-04a0b78e5f75 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:23.924824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:23.925542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:23.925936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:23.926771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:23.927241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:23.927631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:23.929764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:23.940189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 78/148] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:23.946531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.077695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f14b4-f54e-465d-9dee-c2721ab52b9f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.109129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:24.120108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.121006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.121413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.121867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.125628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.127178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.128992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 79/149] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:23 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:24.130395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca76e6c-90e8-4a6e-bdd9-ed5e6271617e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.137935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.190601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.191216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.191504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.191791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.192598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:24.192961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.194304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.209048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 71/150] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:24.220002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.330823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f764e3-9463-47cf-8c52-963b2e6a00af None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.354156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:24.388822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.389753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.390057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.390342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.424361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da27b96-9a2f-4c9c-bb60-834fcd13f842 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.457587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:24.489954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.491579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.504603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.505740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.506202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.506621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] PATH_INFO: `/v3/users/f11416539ba24a47916d4766fdffd7ce` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.509728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] RBAC: Authorizing `identity:get_user(user_id=f11416539ba24a47916d4766fdffd7ce)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.511871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.514704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 72/151] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:24 2022] GET /identity/v3/users/f11416539ba24a47916d4766fdffd7ce => generated 289 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:24.520312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.644891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 80/152] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:24.656460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.713458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11ebf0-0a82-4b4e-b8eb-382f5d76eb17 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.758970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:24.769900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.771336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.771872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.772350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.773879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.775574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.778069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 73/153] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:24 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:24.784762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:24.832320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:24.863054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d58eb7-97b1-477f-8506-79ce9543d531 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:24.915267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:24.916198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:24.916495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:24.916803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:24.918082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:24.920237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:24.922199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 81/154] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:24 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:24.928293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:25.005488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58661886-77c1-4f0a-9a68-ff77c9a151e5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:25.084471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1da598-ee40-4f1f-a296-5696b2d525ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:25.103275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:25.104550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:25.105115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:25.105590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/f11416539ba24a47916d4766fdffd7ce/roles/7dd633f5c336427390fa93ce9c2d4053` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:25.108515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, user_id=f11416539ba24a47916d4766fdffd7ce, role_id=7dd633f5c336427390fa93ce9c2d4053)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:25.110215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa479a-77e7-49ed-a20f-1915a6a1ee73 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:25.139607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:25.140327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:25.140623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:25.140907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:25.141775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:25.142094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:25.143584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239809f-7b82-4dd6-9e72-07a9ab285322 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:25.145500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 74/155] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:24 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/users/f11416539ba24a47916d4766fdffd7ce/roles/7dd633f5c336427390fa93ce9c2d4053 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:25.150537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 82/156] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:24 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=b82b050f41064700b9690d91633b2c52 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:25.913578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb78e9-5fed-49c2-8b13-09e7abb5b340 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:25.914451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb78e9-5fed-49c2-8b13-09e7abb5b340 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:25.914912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb78e9-5fed-49c2-8b13-09e7abb5b340 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:25.916503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 75/157] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:25.923030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffb4a-445e-4b7a-b958-cd1b68139a73 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:25.923864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffb4a-445e-4b7a-b958-cd1b68139a73 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:25.924332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddffb4a-445e-4b7a-b958-cd1b68139a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:25.948791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ddffb4a-445e-4b7a-b958-cd1b68139a73 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:25.950988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ddffb4a-445e-4b7a-b958-cd1b68139a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:25.954312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:26.034101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 83/158] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:26.065901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8b4f5-7c31-4f67-a532-c91d87590236 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.066190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8b4f5-7c31-4f67-a532-c91d87590236 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.066476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8b4f5-7c31-4f67-a532-c91d87590236 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.067859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 76/159] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:26.073678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.073973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.074258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.086569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8b4f5-7c31-4f67-a532-c91d87590236 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.102739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:26.104220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:26.188366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:26.241608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 84/160] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:26.259016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8b4f5-7c31-4f67-a532-c91d87590236 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:26.315284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:26.316120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.316406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.316696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.325014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.334248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:26.343296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:26.344264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:26.344998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:26.349050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 77/161] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:26.355181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.520876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30840697-66bf-49f8-9f95-7e4d29ac36c9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:26.523863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:26.548172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:26.573827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea61f7-a712-47c6-88da-d3d21902699e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:26.577248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:26.578124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.578557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.578975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.579907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:26.580411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:26.581951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:26.599791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 78/162] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:26.617825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.620025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:26.620963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.621417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.621847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.635586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:26.644854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:26.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:26.646865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:26.649578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 85/163] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:26.654439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.816016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894904e5-b694-485d-afd0-745af55fe3bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:26.843193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce6390-73b8-42e7-9ee0-ff88d8233ba7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:26.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:26.885435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.885723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.886005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.887362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:26.888948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:26.890913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 79/164] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:26.898841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:26.920217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:26.924714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:26.925507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:26.925860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:26.926167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:26.927116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:26.927453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:26.930512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:26.947459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 86/165] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:26.953567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.002662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.085849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b1c0-b3b1-4fec-8343-e9abfb5a324a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.146655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.147321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.147613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.147917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.170937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70301e15-96f0-42a7-bdf1-7fda1908dcc5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.234559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.235452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.235739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.236045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] PATH_INFO: `/v3/users/bf9b7c7fda6947388c02a47587519de9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.238480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] RBAC: Authorizing `identity:get_user(user_id=bf9b7c7fda6947388c02a47587519de9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:27.239981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7f5964-8d76-42d7-a06c-e9b48574a8fe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:27.241628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 87/166] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/users/bf9b7c7fda6947388c02a47587519de9 => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:27.250708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167fb2-c65a-4572-82c4-15763a0c47ca None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.251051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167fb2-c65a-4572-82c4-15763a0c47ca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.251338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff167fb2-c65a-4572-82c4-15763a0c47ca None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.252933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 88/167] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:27.255044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff167fb2-c65a-4572-82c4-15763a0c47ca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.281315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:27.282812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d3bb2-d4a4-473c-ae12-0ddcfd9293f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:27.352869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.374913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.426072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff167fb2-c65a-4572-82c4-15763a0c47ca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.460898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.461558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.461842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.462117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.463555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 80/168] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:27.465377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.465935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.466506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.497366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:27.499061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:27.532346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:27.533822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:27.596329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.640553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 89/169] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:27.641671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.655638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 81/170] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:27.659035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.766555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcb58e-95a0-4d04-a531-f6756bbe3942 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.802182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.803349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.803808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.804282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.805679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:27.807359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:27.809333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 90/171] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:27 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:27.810270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.810954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.838339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:27.851901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee88d341-c954-4f46-9d26-80c943e4e6ab None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.909360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.910160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.910462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.910915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] PATH_INFO: `/v3/projects/0cfa61b3cc4f4aa48a071128544d246e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.912275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] RBAC: Authorizing `identity:get_project(project_id=0cfa61b3cc4f4aa48a071128544d246e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:27.913750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5568d3-236c-47a2-b2d3-7d076a68b4bd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:27.915070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d7c7a-03e0-4343-b874-fb7cc5e10ac2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:27.915458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 82/172] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:27 2022] GET /identity/v3/projects/0cfa61b3cc4f4aa48a071128544d246e => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:27.916799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5969-768c-41b1-8281-7107d63691c6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.917195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5969-768c-41b1-8281-7107d63691c6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.917479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c5969-768c-41b1-8281-7107d63691c6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.919691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 83/173] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:27.920574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c5969-768c-41b1-8281-7107d63691c6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:27.953199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:27.954915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.956356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.956666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.970363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-77f1cc90-4d61-486b-a467-ac051d0e3651 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:27.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:27.982330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:27.985617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 91/174] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:27.988476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9544-41a9-4734-8c46-acb9a462a7ef None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.988476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9544-41a9-4734-8c46-acb9a462a7ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.988738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d9544-41a9-4734-8c46-acb9a462a7ef None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:27.990030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 92/175] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:27.991243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:27.992909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:27.992909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.013450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:28.014672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:28.064381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c5969-768c-41b1-8281-7107d63691c6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.107574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 93/176] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:28.108018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.108695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.109037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.109324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.109628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.110213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:28.110490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:28.111975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:28.118216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 84/177] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:27 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:28.119222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.130428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.132880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.222434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a884e-5f36-40fc-86e0-3a78468e51a8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.242730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364dbb4-eb62-4ba8-9087-93b9924f6707 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.261939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.262957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.263532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.264034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.265197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=0cfa61b3cc4f4aa48a071128544d246e, user.id=bf9b7c7fda6947388c02a47587519de9) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:28.265726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:28.267483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce4a9ae-3122-4a7f-8f5e-9f54aa1d6206 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:28.275729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 94/178] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/role_assignments?user.id=bf9b7c7fda6947388c02a47587519de9&scope.project.id=0cfa61b3cc4f4aa48a071128544d246e&role.id=b82b050f41064700b9690d91633b2c52 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:28.277394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67486-f69a-407c-ab8a-64abe3269ba2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.278448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67486-f69a-407c-ab8a-64abe3269ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.278955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67486-f69a-407c-ab8a-64abe3269ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.305367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8cd67486-f69a-407c-ab8a-64abe3269ba2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:28.307047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cd67486-f69a-407c-ab8a-64abe3269ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:28.316566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.317734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.317734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.317899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.318582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:28.319236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:28.321666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:28.340794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 85/179] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:28.341677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.423882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 95/180] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:28.425733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb91f-a329-4809-87bc-58417720ebdf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.426261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb91f-a329-4809-87bc-58417720ebdf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.426785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fb91f-a329-4809-87bc-58417720ebdf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.428364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 96/181] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:28.429313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fb91f-a329-4809-87bc-58417720ebdf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.436654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.480250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10085-ec72-41bb-8154-710f9b425192 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.480453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.553906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.554713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.554998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.555277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.570207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-af167d7c-94f1-4abf-b8c6-29618020cb85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:28.580824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:28.581736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:28.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 86/182] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:28.583640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.583640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.583801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.586196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fb91f-a329-4809-87bc-58417720ebdf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.605379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:28.606608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:28.639557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.640820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.641100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.641728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] PATH_INFO: `/v3/users/a064d7a4fdb343efa665d8a825cdfba7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.643017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] RBAC: Authorizing `identity:get_user(user_id=a064d7a4fdb343efa665d8a825cdfba7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:28.644529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:28.646178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 97/183] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/users/a064d7a4fdb343efa665d8a825cdfba7 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:28.646925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.704405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 87/184] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:28.705354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.764961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.771472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:28.785176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568942ff-c2d7-42a8-8b9f-e32764685491 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.813955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84e85e-002c-4742-8e03-9d09ddfb9e3d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.820720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.821621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.821903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.822186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.833514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:28.842222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:28.843095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 98/185] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:28.848446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.852267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:28.852950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:28.853659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:28.853945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:28.854706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:28.855366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:28.857579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:28.870945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 88/186] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:28.870945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:28.968220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb5069-93b8-44ee-974e-b71ca330dfc6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:28.985904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bfae6-31dd-456b-b9c4-bfe780bae937 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.006741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.007401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.007692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.008032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.012928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.020950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.025524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.026390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.026676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.026960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.028218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.029668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.035700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 89/187] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.036547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.082358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.083853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.142920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6d1c4-69f8-4fe7-b5ed-7c1120302c0e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.176064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.176789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.177086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.177371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.178127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:29.178442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.179811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.181814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 99/188] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:29.182659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.193398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 90/189] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.193398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.226641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.247529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.295067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67533d-4e67-418b-b9cc-e17fb541e00c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.295562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4705e-ff9d-4531-9ade-e9cd67be246a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.327990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.328199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.328665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.328943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.329033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.329226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.329313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.329500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.329948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.331393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.343051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.344556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311e33f6-ce69-4dfe-acc7-8f66e93f7277 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.346294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 100/190] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.347494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6603a-2e93-43cc-958d-567cbe6cdb4c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.347899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6603a-2e93-43cc-958d-567cbe6cdb4c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.348088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6603a-2e93-43cc-958d-567cbe6cdb4c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.349575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 101/191] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:29.350139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6603a-2e93-43cc-958d-567cbe6cdb4c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.353919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 91/192] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:29 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:29.360977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.465550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.471712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6603a-2e93-43cc-958d-567cbe6cdb4c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.481165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b863c-9855-4a87-b2cf-fa6396ae286a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.497888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.513171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.514574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.514890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.515181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.516509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.518016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.519793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 102/193] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.520611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.538597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.539580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.539968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.540335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.541729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.543232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d968965-30f5-4a07-8d7e-944ca5eb18df None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.550256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 92/194] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.550256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.550256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.550256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.572939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:29.574163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:29.645375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9e110-0798-4ce2-9323-942f938d2c99 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.654795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 93/195] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:29.655744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.686167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.686829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.687117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.687393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.732307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.781451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.783493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.787781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803194a-127d-45fa-9173-4421b5ff7a40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.799975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:29.830325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:29.830969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:29.831259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:29.831635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:29.832567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=a064d7a4fdb343efa665d8a825cdfba7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:29.832884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:29.834496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:29.841807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 94/196] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/role_assignments?user.id=a064d7a4fdb343efa665d8a825cdfba7&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:29.842626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:29.969680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9983f36-4d25-42bc-a442-07a9d8e2ed55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:29.983064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 103/197] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:29.983956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.016856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.017970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.018268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.018713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.096307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.122769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.133682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.135166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.162194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197d7b8a-514b-41d7-8432-f9e7a4c8ff6c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.198378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.200050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.200585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.201066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.212172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:30.220353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:30.221203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:30.221996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:30.224422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 104/198] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:30.225660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.243314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 95/199] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:30.248055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.319632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.342796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34992b-599a-4b02-a169-ebe1a5363c83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.355262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ec3ba-dcef-4807-a46f-9fd64a89fbdc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.372421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.379708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.380891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.381284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.381857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.383075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.384959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.386812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 105/200] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.392338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.396349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.397193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.397779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.398250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.399209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:30.399715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.401317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.410677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 96/201] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.411810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.509566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33751e92-4d50-465c-9723-4ceadcbc5485 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.516876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785d9c8-e133-4b8e-857f-0678fe244b43 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.545831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.546735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.547101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.547303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.548579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.550170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.552483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 106/202] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.556515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.558008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.559394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.560304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.560780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.561823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=a1a088e94b7d4d97af17a3b074da3828) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:30.562333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.564036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.566130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.570627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 97/203] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/role_assignments?user.id=a1a088e94b7d4d97af17a3b074da3828&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.571597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.712444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:30.797730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b377881-755a-4e42-a03c-e8c7f197a238 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.818236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4979fc7-cc22-4324-8c2a-692c7ca885b2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:30.869204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.869873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.870165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.870455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.871896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=a1a088e94b7d4d97af17a3b074da3828) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:30.872242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.873818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.880654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 98/204] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/role_assignments?user.id=a1a088e94b7d4d97af17a3b074da3828&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.881221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:30.882449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:30.882746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:30.883026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] PATH_INFO: `/v3/users/f11416539ba24a47916d4766fdffd7ce` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:30.893063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] RBAC: Authorizing `identity:get_user(user_id=f11416539ba24a47916d4766fdffd7ce)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:30.894593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:30.896441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 107/205] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/users/f11416539ba24a47916d4766fdffd7ce => generated 289 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:30.902206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:30.987047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:31.048984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeac059-39f9-47eb-b6a7-81bad64513fa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:31.096761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:31.097440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.097732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.098020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.182403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:31.184517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:31.255499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:31.312673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 99/206] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:31.318093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:31.452181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c3096-61ae-478d-8ffd-1fd1203b24a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:31.481314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:31.497672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:31.498982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.499646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.500417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.501712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:31.503195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c2f38-4f53-43c5-9f45-bc32b3001ea6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:31.504946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 108/207] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:31 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:31.510923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:31.663568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e23dd-0477-452d-9b68-8f2bdbd089e6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:31.700301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:31.701419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.701982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.702454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.703486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7, user.id=f11416539ba24a47916d4766fdffd7ce) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:31.704028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:31.705694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4aa2-6b37-4c60-b722-8cab898e7451 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:31.713241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 100/208] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:31 2022] GET /identity/v3/role_assignments?user.id=f11416539ba24a47916d4766fdffd7ce&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:31.954445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54604-568e-48f9-9cac-75934750cb40 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.955108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54604-568e-48f9-9cac-75934750cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.955608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54604-568e-48f9-9cac-75934750cb40 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.957244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 109/209] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:31.964200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08299d3d-b4be-4ea6-99e9-d01d27ce7dd0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:31.964784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08299d3d-b4be-4ea6-99e9-d01d27ce7dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:31.965335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08299d3d-b4be-4ea6-99e9-d01d27ce7dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:31.984015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-08299d3d-b4be-4ea6-99e9-d01d27ce7dd0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:31.985520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-08299d3d-b4be-4ea6-99e9-d01d27ce7dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:32.009878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:32.067910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 101/210] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:32.092124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54604-568e-48f9-9cac-75934750cb40 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.204208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54604-568e-48f9-9cac-75934750cb40 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.228268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1d581-cb7d-4e13-9a58-8ed59e75f08b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.229044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1d581-cb7d-4e13-9a58-8ed59e75f08b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.229543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1d581-cb7d-4e13-9a58-8ed59e75f08b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.232375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 102/211] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:32.247264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.247898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.248391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.250178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.250988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.251265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.251544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.269607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:32.271638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:32.272877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:32.282819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:32.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:32.284530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:32.285118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 110/212] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:32.291880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.364260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:32.371839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 103/213] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:32.392086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.413829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f15e53-9df3-47fe-8094-d189ca08fe62 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.454381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.455604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.456193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.456685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.457695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:32.458296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:32.460038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:32.469853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 111/214] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:32.476022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.510188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:32.530948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbffe5d6-9d71-4d65-add0-10a02133e322 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.576809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.577622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.577910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.578194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.580023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:32.582500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:32.589165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 104/215] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:32.590163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.606407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c9ff-4ed6-4cd9-852d-706e2b8e27f7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.634608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:32.657915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.659452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.660212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.661086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] PATH_INFO: `/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.680806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] RBAC: Authorizing `identity:get_user(user_id=c0f0221f64ff4ffcb222a5b58b249cd1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:32.683273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:32.686545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 112/216] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:32 2022] GET /identity/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:32.692932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:32.710795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220c48e-37c1-4e63-b91a-175a66340581 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.746612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.747297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.747599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.747926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.748654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:32.750105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:32.780122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 105/217] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:32 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:32.815420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f11c-caa4-481e-949a-046eb4eb52bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:32.856241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:32.857315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:32.857826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:32.858303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:32.912886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:32.941731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:32.943643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7551d-309c-4925-acb2-4157e54d4e4e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.054122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 113/218] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:33.060865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.079436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:33.132118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe561-19f5-4913-8628-ab8f3bfbef31 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.132757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe561-19f5-4913-8628-ab8f3bfbef31 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.133183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fe561-19f5-4913-8628-ab8f3bfbef31 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.135276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 114/219] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:33.141515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.142005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.142428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.163162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:33.164858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:33.198211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364dc83-17fa-4542-918d-350adcd65ec0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.238832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.239717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.240211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.240568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.242318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:33.243460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.244034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 115/220] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:33.245508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 106/221] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:33.251485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.263137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:33.268237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.366065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:33.379427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742d676-9d29-4a1e-8f18-40463f71843f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.396752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88c624-a44e-4615-a7f6-3a8d5a79f84a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.430056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.431030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.431493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.432031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.433168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=c0f0221f64ff4ffcb222a5b58b249cd1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:33.433744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:33.435997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.443836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.443836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.443836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.443836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.454415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 116/222] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/role_assignments?user.id=c0f0221f64ff4ffcb222a5b58b249cd1&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=b82b050f41064700b9690d91633b2c52 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:33.456585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9ca9261d-c781-411e-8739-a41ce429374d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:33.464733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:33.465757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:33.466554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:33.467143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 107/223] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:33.472514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.577849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:33.613062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0bc49-4c80-477f-95c7-7e96213b66dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.641892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cba49-ea96-46cb-a774-99ac2c050b2d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.642510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cba49-ea96-46cb-a774-99ac2c050b2d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.642988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cba49-ea96-46cb-a774-99ac2c050b2d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.644890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 108/224] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:33.654293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.654920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.655458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.670115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.670811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.671099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.671382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.672182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:33.672499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:33.673931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.681633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:33.682865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:33.685034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 117/225] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:33.692823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.785330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 109/226] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:33.806439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.877618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5cb6b0-9efe-4252-b86e-d7cdede5a39d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:33.918801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:33.919600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:33.919910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:33.920200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:33.927101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:33.928633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:33.930313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 118/227] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:33.937114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:33.942982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:33.958342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:34.025986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b636fbb-d62d-4625-9873-14f96a04e6d3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.080686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac1bda-7f96-494e-b30a-047164da030f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.090245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.091523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.092037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.092533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.108319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:34.118095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:34.118940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:34.119703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:34.131610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 110/228] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:34.140078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:34.185182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.185991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.186281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.186564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.187787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.189302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.191022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 119/229] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:33 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:34.196787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:34.249526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f08119-aa92-4be6-9c20-0d38d474ab50 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.280237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:34.288955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.289870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.290348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.290818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.291766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:34.292357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.294457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.304571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 111/230] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:34.312977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:34.389699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:34.435879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ec5ae-c469-4920-bf15-74e4ec7a628e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.482161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634cc6-3027-45f2-b961-8e497cf63ccf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.489424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.490486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.490663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.490896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/users/a1a088e94b7d4d97af17a3b074da3828/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.492964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, user_id=a1a088e94b7d4d97af17a3b074da3828, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.495744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.526196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 120/231] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:34 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/users/a1a088e94b7d4d97af17a3b074da3828/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:34.533793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.535195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.535720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.536371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.537999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.539897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.541899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 112/232] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:34 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:34.549023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:34.700736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55da39d-4cc1-4ca5-b98a-130433a06e8c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.732257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:34.745756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.747305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.747874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.748740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.750119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.752447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d982a-5588-4e37-996d-879b44537e3b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.753848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 121/233] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:34 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:34.759698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:34.832240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:34.870882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05c9fe-17d1-49ea-b099-8bc713071e6d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:34.906792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:34.907589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:34.907907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:34.908199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/users/a1a088e94b7d4d97af17a3b074da3828/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:34.910624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, user_id=a1a088e94b7d4d97af17a3b074da3828, role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:34.912160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a06014-01c2-4854-aa1a-ae8a3cd858a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:34.930260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 113/234] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:34 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/users/a1a088e94b7d4d97af17a3b074da3828/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:35.492866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a2d3-6a8f-46fa-b1b9-768a5df23822 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:35.493404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a2d3-6a8f-46fa-b1b9-768a5df23822 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:35.493404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0a2d3-6a8f-46fa-b1b9-768a5df23822 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:35.495069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 122/235] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:35.501081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:35.501538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:35.501832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:35.528677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:35.529878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:35.613332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 114/236] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:35.638286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0a2d3-6a8f-46fa-b1b9-768a5df23822 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:35.775388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0a2d3-6a8f-46fa-b1b9-768a5df23822 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:35.810086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:35.810900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:35.811187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:35.811469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:35.821951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:35.829715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:35.831837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:35.833911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 123/237] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:35.840257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:35.927413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.015072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337461f-7322-40a2-b7d7-1a0bbbb23690 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.078797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.079413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.079699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.080005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.080793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:36.081101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:36.082532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:36.096334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 115/238] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:36.101796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.181316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.224102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88187af-7850-4e79-ad4c-a2a81d5926d7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.266660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.267686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.268001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.268282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] PATH_INFO: `/v3/users/f06d0102041544d89fd381a0478e7923` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.286007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] RBAC: Authorizing `identity:get_user(user_id=f06d0102041544d89fd381a0478e7923)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:36.287485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e2dfd-8e6b-4402-be95-8eff575b17b6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:36.289585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 124/239] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/users/f06d0102041544d89fd381a0478e7923 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:36.295507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.307413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0aed2-7210-41e1-a194-883174fc4f8a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.308049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0aed2-7210-41e1-a194-883174fc4f8a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.309163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f0aed2-7210-41e1-a194-883174fc4f8a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.311138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 125/240] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:36.316766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5394c2-6878-46c8-970a-2e65ab8ebc39 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.317284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5394c2-6878-46c8-970a-2e65ab8ebc39 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.317787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5394c2-6878-46c8-970a-2e65ab8ebc39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.339577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ba5394c2-6878-46c8-970a-2e65ab8ebc39 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:36.341868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ba5394c2-6878-46c8-970a-2e65ab8ebc39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:36.428280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 126/241] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:36.429331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721714a-760b-417d-a4d2-9b61dfd40ba8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.429630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721714a-760b-417d-a4d2-9b61dfd40ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.429910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721714a-760b-417d-a4d2-9b61dfd40ba8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.437702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 127/242] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:36.438727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.439021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.439300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.467650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82d45e-c0ae-4b13-969b-12c2a536fa1a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.467871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:36.469627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:36.479881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.517437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.518773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.519243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.519659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.530954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.559191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 128/243] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:36.560373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.626006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:36.628155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:36.687563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d12c4-e3e9-414e-9076-82db1173286d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.724279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.725278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.725673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.726055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.727353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:36.729816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:36.732704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 129/244] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:36.738013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.766226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 116/245] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:36.767084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.784776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.800731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:36.869895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e13037-bd0c-4ca7-b71d-4ac960e2049f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.884127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a911-ba25-4ef7-99b4-9a95a2dc002a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:36.908889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.909752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.910041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.910392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.923955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:36.924451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:36.924903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:36.925294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:36.925678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:36.926523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:36.928140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:36.932689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:36.935077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 130/246] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:36.937011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:36.953624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 117/247] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:36 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:36.961354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.054798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b960b-9475-4afd-a358-29c24bc0d673 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.068250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.095062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.095994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.096283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.096563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.097941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.099621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.101548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 131/248] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:37.109340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.131678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.185691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddda5-1dd0-490b-96ed-1c8e3fd69003 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.234493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.235328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.235661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.236087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.237120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:37.237445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.238936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815b10c-1730-4a5e-b851-1ee37323516a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.249250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739f936-fb11-4c4e-802e-903407ec1051 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.250469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 118/249] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:37.252294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76e1e5-9671-4fdf-a00c-5ac21d53adc4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.252430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76e1e5-9671-4fdf-a00c-5ac21d53adc4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.252705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f76e1e5-9671-4fdf-a00c-5ac21d53adc4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.254197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 119/250] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:37.259425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76e1e5-9671-4fdf-a00c-5ac21d53adc4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.287995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.288723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.289018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.289338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.290153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=f06d0102041544d89fd381a0478e7923) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:37.290470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.293416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eb7d3-cd47-44bb-a0a4-e35980c40319 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.299646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 132/251] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/role_assignments?user.id=f06d0102041544d89fd381a0478e7923&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:37.300833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.301128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.301411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.322900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:37.324386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:37.364864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.386902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f76e1e5-9671-4fdf-a00c-5ac21d53adc4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.417599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 133/252] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:37.427014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.430813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.432056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.432416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.432790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] PATH_INFO: `/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.434913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] RBAC: Authorizing `identity:get_user(user_id=c0f0221f64ff4ffcb222a5b58b249cd1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.436760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.439893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 120/253] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:37.446101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.446745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.570676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88df3a1-a07e-4204-be69-51a4a12c7e2e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.615633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df92ade0-b711-4a31-b7aa-e00953d33ff9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.617715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.620186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.620610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.639858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:37.651463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:37.652731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:37.653839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 121/254] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:37.660977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.678289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.679356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.679923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.680425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.682833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.684685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.691202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 134/255] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:37.692346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.723901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.739996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:37.817616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d4d39-7833-40a4-8afd-c4b8e90298a7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.888227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e188ae2-4a19-4ea4-90d3-9480c2dde2fd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:37.891201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.891967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.892287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.892573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.894278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:37.894613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.896091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab1bd4-1c90-428b-9877-128732820b67 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.911195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 122/256] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:37.913728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edc59b-e5ec-45a1-ad84-90982c529240 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.914028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edc59b-e5ec-45a1-ad84-90982c529240 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.914316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edc59b-e5ec-45a1-ad84-90982c529240 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.918374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 123/257] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:37.922412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edc59b-e5ec-45a1-ad84-90982c529240 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:37.950190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:37.950986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.951276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.952401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/users/c0f0221f64ff4ffcb222a5b58b249cd1/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:37.955206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, user_id=c0f0221f64ff4ffcb222a5b58b249cd1, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:37.957353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58790e2-d9a5-4d48-b099-40d142a1fbcb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:37.982738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 135/258] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:37 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/users/c0f0221f64ff4ffcb222a5b58b249cd1/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:37.985171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:37.985471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:37.985758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.019252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:38.021509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:38.132722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edc59b-e5ec-45a1-ad84-90982c529240 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.152814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 136/259] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:38.161013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.180511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.197279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.198181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.198468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.198744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.203923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.205369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.207140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 124/260] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:37 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:38.212488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.230157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.380241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20240c10-9eed-4bd2-bd49-2bbd5ab1b46e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.426039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a0a67-081c-44eb-a32a-5392595ef66d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.431018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.431866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.432313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.432765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.443971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:38.453015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:38.454210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:38.455152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:38.457354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 137/261] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:38.481056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.486385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.487056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.487344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.487625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.576483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.587303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.598076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60836b-192f-47f3-b19e-7c24b034533e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.611274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.616879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.636023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.636934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.637227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.637507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.638284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:38.638594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.640268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.650238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 138/262] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:38.662485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.762015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 125/263] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:38.767639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.784192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1456fd1-34e6-4390-8dbc-5f6289cff1bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:38.856299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:38.857667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:38.857955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:38.858236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] PATH_INFO: `/v3/users/a1a088e94b7d4d97af17a3b074da3828` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:38.860417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] RBAC: Authorizing `identity:get_user(user_id=a1a088e94b7d4d97af17a3b074da3828)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:38.861915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:38.863594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 139/264] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/users/a1a088e94b7d4d97af17a3b074da3828 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:38.873314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:38.895029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.944940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:38.982017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff734883-c4a8-4335-93be-5e58738879a7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.009950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a9d8dd-f9e6-4946-8e38-c0731834181a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.046448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.047319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.047770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.048219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.080531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.081416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.081713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.081997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.083227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.084717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.086666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 126/265] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:39.092782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.123023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.124819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.141168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:39.239582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 140/266] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:39.245017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.305593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0088a-229d-431a-a8d1-d12ae31d1828 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.381755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.382428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.382718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.383002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.384133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=a1a088e94b7d4d97af17a3b074da3828) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:39.384451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.385913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.408475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 127/267] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:39 2022] GET /identity/v3/role_assignments?user.id=a1a088e94b7d4d97af17a3b074da3828&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:39.443805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67634-308e-4fd1-9362-4bb909e1eaf7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.484206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:39.501474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.502368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.502661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.502947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.505068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.506539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aecd61b-8103-4228-9071-e86aac94f659 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.508717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 141/268] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:39 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:39.514575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:39.543698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:39.658625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7da116-e30b-4b34-ad23-19d1b4fb47f3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:39.703860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:39.704552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:39.704841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:39.705121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:39.705978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=a1a088e94b7d4d97af17a3b074da3828) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:39.706296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:39.707796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d0b06-17f6-4d61-a235-196f2e9e7428 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:39.715331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 128/269] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:39 2022] GET /identity/v3/role_assignments?user.id=a1a088e94b7d4d97af17a3b074da3828&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:40.288445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa151f8-984d-4f64-9ad4-2ba19d8f81a7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.289471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa151f8-984d-4f64-9ad4-2ba19d8f81a7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.289978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa151f8-984d-4f64-9ad4-2ba19d8f81a7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.291545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 142/270] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:40.297475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288052b9-ab4e-4716-b585-14897eaa42ac None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.298091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288052b9-ab4e-4716-b585-14897eaa42ac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.298629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288052b9-ab4e-4716-b585-14897eaa42ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.317659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-288052b9-ab4e-4716-b585-14897eaa42ac None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:40.319195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-288052b9-ab4e-4716-b585-14897eaa42ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.395748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 129/271] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:40.410541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61251f0-17c2-4e13-a8a4-eb4af91f962a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.411165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61251f0-17c2-4e13-a8a4-eb4af91f962a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.411669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61251f0-17c2-4e13-a8a4-eb4af91f962a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.413274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 143/272] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:40.418746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3153319-4367-44fa-b961-cb9c657aa815 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.419359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3153319-4367-44fa-b961-cb9c657aa815 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.419887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3153319-4367-44fa-b961-cb9c657aa815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.426265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61251f0-17c2-4e13-a8a4-eb4af91f962a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.437986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b3153319-4367-44fa-b961-cb9c657aa815 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:40.439593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b3153319-4367-44fa-b961-cb9c657aa815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.448209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.517982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 130/273] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:40.519458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8772-6d0b-4225-8a8b-aa0c450257a3 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.519844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8772-6d0b-4225-8a8b-aa0c450257a3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.520205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd8772-6d0b-4225-8a8b-aa0c450257a3 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.521545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 131/274] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:40.526788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.527119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.527607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.544451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:40.545662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:40.563783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61251f0-17c2-4e13-a8a4-eb4af91f962a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.601260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.602035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.602323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.602604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.614520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:40.616987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 132/275] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:40.617959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:40.623948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:40.624796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:40.625558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 144/276] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:40.632076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.681827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.703121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.737551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bec3a-dac9-48a8-9b83-3538ab68d615 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.744919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ee9889-f817-4f86-b839-5e1cc16ddf41 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.772105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.772897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.773180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.773451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.774608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.776081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.777851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 133/277] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:40.778607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.786091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.786767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.787243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.787672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.788608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:40.789076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.790469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.804570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 145/278] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:40.805668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.880979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d75f4-e936-4341-8992-7467ad008840 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.913297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc643077-736a-4836-83c4-eddc697b33a2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:40.916762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.923807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.924654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.924943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.925227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.935074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:40.938163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:40.946351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:40.947395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:40.948409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:40.950716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 134/279] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:40.951734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:40.957069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:40.957761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:40.958050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:40.958328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:40.959053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:40.960552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:40.983437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 146/280] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:41:40 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:40.984295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.086411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a2064-71f3-4e4c-a5fc-d8a5de23c481 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.098810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17206a06-b74b-412c-a4a2-d16b5ab7f102 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.128910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.130341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.130839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.131371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] PATH_INFO: `/v3/users/4ba83c029874425799becae9fe492891` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.136260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.137520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.137998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.138457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.139471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:41.140017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.141717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.151019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] RBAC: Authorizing `identity:get_user(user_id=4ba83c029874425799becae9fe492891)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.151660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 147/281] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:41.153564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.155775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 135/282] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:40 2022] GET /identity/v3/users/4ba83c029874425799becae9fe492891 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:41.158370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.164911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.186500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.244268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.284981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b357f9-fee0-48d6-8ddd-682ce0b60677 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.304795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028c738-e753-4cf3-8e70-b729f7c07a7d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.326279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.327293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.327585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.329510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] PATH_INFO: `/v3/users/f06d0102041544d89fd381a0478e7923` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.331083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] RBAC: Authorizing `identity:get_user(user_id=f06d0102041544d89fd381a0478e7923)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.332758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.335886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 148/283] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/users/f06d0102041544d89fd381a0478e7923 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:41.343213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.356231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.357098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.357483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.357864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.358823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, user.id=4ba83c029874425799becae9fe492891, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:41.359243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.361817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843dc7c-8bb0-4311-b542-03b2736ef6b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.372729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 136/284] 10.222.0.24 () {56 vars in 1308 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/role_assignments?user.id=4ba83c029874425799becae9fe492891&scope.system=all&role.id=b82b050f41064700b9690d91633b2c52 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:41.465572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.472867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8dfcb-d3a2-4b95-92d7-194d7d6b1083 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.473158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8dfcb-d3a2-4b95-92d7-194d7d6b1083 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.473675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8dfcb-d3a2-4b95-92d7-194d7d6b1083 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.475577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 137/285] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:41.482502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238a416-ff4c-4996-942d-8b09e4839660 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.487531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.488179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.488694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.524167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:41.526276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:41.547570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.548545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.548832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.549109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.550411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.551991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.554113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 149/286] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:41.561263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.645749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 138/287] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:41.685378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:41.691777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dccb87-69e6-4382-b67b-83d65323bbc6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.733076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.733982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.734272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.734558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/users/f06d0102041544d89fd381a0478e7923/roles/7dd633f5c336427390fa93ce9c2d4053` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.737947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, user_id=f06d0102041544d89fd381a0478e7923, role_id=7dd633f5c336427390fa93ce9c2d4053)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:41.739421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:41.767810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 150/288] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:41:41 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/users/f06d0102041544d89fd381a0478e7923/roles/7dd633f5c336427390fa93ce9c2d4053 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:41.794035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:41.861989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9961a77-e4f4-4ade-8685-498988d83d58 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:41.907696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:41.909018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:41.909510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:41.910025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:41.934702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:41.942874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:41.943737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:41.949107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:41.951885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 139/289] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:41.957686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.007046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:42.122556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fba91-58d2-449a-bd69-4f535eb55cf4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.162678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.163286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.163576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.163879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.164650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:42.164965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.166366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.175597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 151/290] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:42.181240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.297798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09dfcf5-4b1b-40bb-87b8-5955fe4bf62d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.311119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:42.334542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.335426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.335713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.336025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] PATH_INFO: `/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.337227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] RBAC: Authorizing `identity:get_user(user_id=c0f0221f64ff4ffcb222a5b58b249cd1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.338680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.340409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 140/291] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:42 2022] GET /identity/v3/users/c0f0221f64ff4ffcb222a5b58b249cd1 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:42.345917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.444585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:42.468969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0761-dbdb-4302-8a24-fa3e42772b07 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.518880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.519553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.519859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.520150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.615488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.617003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.730598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:42.758766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 152/292] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:42.764510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:42.916081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93497e64-0665-43eb-a62c-953096a71d2a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:42.951778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4865aec3-d989-4b29-a696-804921666055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:42.952683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865aec3-d989-4b29-a696-804921666055 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:42.952972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865aec3-d989-4b29-a696-804921666055 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:42.953255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4865aec3-d989-4b29-a696-804921666055 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:42.954469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865aec3-d989-4b29-a696-804921666055 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:42.955998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4865aec3-d989-4b29-a696-804921666055 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:42.957796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 141/293] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:42 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:42.964030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.104009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3f8b3-02d6-4d9c-b56c-26401378af29 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:43.154631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:43.155346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.155643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.155994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.156843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=c0f0221f64ff4ffcb222a5b58b249cd1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:43.157165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:43.158621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ac15d-042c-4ad4-8727-bb4a6973e7c5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:43.169297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 153/294] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:42 2022] GET /identity/v3/role_assignments?user.id=c0f0221f64ff4ffcb222a5b58b249cd1&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=b82b050f41064700b9690d91633b2c52 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:43.732698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120be9-a051-46a6-b487-102c73804321 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.733386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120be9-a051-46a6-b487-102c73804321 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.733911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120be9-a051-46a6-b487-102c73804321 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.735719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 142/295] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:43.741479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:43.742352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:43.742834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:43.763199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:43.764750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:43.812571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:43.846773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 154/296] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:43.873934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120be9-a051-46a6-b487-102c73804321 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:43.938878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.002810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120be9-a051-46a6-b487-102c73804321 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.038779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.039506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.039800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.040107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.051760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-71ab67f0-b675-4423-8664-ffc0c8ed5a62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:44.060488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:44.061389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:44.062403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 143/297] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:44.067479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.088344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506469-6989-4a5e-ba06-4b063bbbd789 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.088952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506469-6989-4a5e-ba06-4b063bbbd789 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.090130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb506469-6989-4a5e-ba06-4b063bbbd789 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.092417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 144/298] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:44.097103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49c43-9502-4592-8290-c8b467df4589 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.097607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49c43-9502-4592-8290-c8b467df4589 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.098281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e49c43-9502-4592-8290-c8b467df4589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.119948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e2e49c43-9502-4592-8290-c8b467df4589 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:44.122208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e2e49c43-9502-4592-8290-c8b467df4589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:44.179867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b88155-82bf-4483-9fbd-0ccfdc164524 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.204053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 145/299] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:44.205507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aeb3-1697-4f5f-baea-df824f26d5c6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.205805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aeb3-1697-4f5f-baea-df824f26d5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.206091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45aeb3-1697-4f5f-baea-df824f26d5c6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.207937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 146/300] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:44.213080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.213598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.214078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.226602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.227279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.227722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.228220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.228987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:44.229297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.230730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.232937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:44.234383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:44.242351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 155/301] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:44.247889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.266628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.292379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.312781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 147/302] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:44.313536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.373373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d25a2-325c-4c34-a4d5-0170b3184c2b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.409458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.410248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.410531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.410806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.412011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.413453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.415040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 156/303] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:44.415956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.422149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f04d42-9db4-4e16-9d3d-d199ed4d3e18 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.458503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.459373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.459661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.459970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] PATH_INFO: `/v3/users/9b4b07a6b61f4b1498764ee50a189953` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.475655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] RBAC: Authorizing `identity:get_user(user_id=9b4b07a6b61f4b1498764ee50a189953)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.477463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.484454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 148/304] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/users/9b4b07a6b61f4b1498764ee50a189953 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:44.485626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.522788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.531574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ba6e-b211-4144-9585-820437054b78 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.535469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.570668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.571853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.572141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.572416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.585910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:44.594241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:44.595130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:44.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:44.598082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 157/305] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:44.599081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.611983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b823208-1bcf-47de-830a-afe83f5e90ee None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.658627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.659299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.659588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.659895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] PATH_INFO: `/v3/groups` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.660587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.662032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.677065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 149/306] 10.222.0.24 () {60 vars in 1134 bytes} [Tue May 24 13:41:44 2022] POST /identity/v3/groups => generated 229 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:41:44.677901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.710963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573a29c-dad6-44fc-a97b-a4bea46a7e24 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.750918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.751601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.751916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.752182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.753040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, user.id=9b4b07a6b61f4b1498764ee50a189953, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:44.753360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.754829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95490-90d6-45a8-bee5-b3c7f72dd2b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.762721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 158/307] 10.222.0.24 () {56 vars in 1308 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/role_assignments?user.id=9b4b07a6b61f4b1498764ee50a189953&scope.system=all&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:44.764127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde033a-46ca-4642-ac86-aaeb674a3c99 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.764426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde033a-46ca-4642-ac86-aaeb674a3c99 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.764710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde033a-46ca-4642-ac86-aaeb674a3c99 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.766420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 159/308] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:44.773434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.773966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.774457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.782297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.796490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:44.798090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:44.800994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bf5e2-8515-45c7-ad77-78852322bad7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:44.829487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:44.841822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:44.842539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:44.842850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:44.843112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:44.843961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:44.844283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:44.845757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:44.856318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 150/309] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:44.862272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.891213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 160/310] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:44.919254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:44.999378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0122904-154c-402c-ab8e-398b75d9140e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.041401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.043293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.043587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.043913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] PATH_INFO: `/v3/users/4ba83c029874425799becae9fe492891` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.045435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] RBAC: Authorizing `identity:get_user(user_id=4ba83c029874425799becae9fe492891)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.047858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.049543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f47c4-24dc-4308-99e3-51ec20ebae09 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.049734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 151/311] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/users/4ba83c029874425799becae9fe492891 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:45.055863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:45.079880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:45.091795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.092768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.093200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.093582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.105484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:45.114233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:45.115333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:45.116145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:45.122148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 161/312] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:45.123311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:45.144089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:45.205951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd541d50-34cb-4b25-8618-d90dd51feb5e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.250012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54fd9f-9db9-40e0-ad53-6fa41e2f7dd6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.254452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.255328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.255612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.256012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] PATH_INFO: `/v3/system/users/4ba83c029874425799becae9fe492891/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.258199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ba83c029874425799becae9fe492891, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.260550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.272889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 152/313] 10.222.0.24 () {58 vars in 1334 bytes} [Tue May 24 13:41:45 2022] PUT /identity/v3/system/users/4ba83c029874425799becae9fe492891/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:45.299744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.300578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.300867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.301146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.301996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:45.302317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.304023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.314768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 162/314] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:45.321168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:45.419175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:45.441021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f29bc8-e1e8-4186-9fe0-44045f2c056b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.479027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.480753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.481364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.481641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] PATH_INFO: `/v3/users/f06d0102041544d89fd381a0478e7923` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.482882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] RBAC: Authorizing `identity:get_user(user_id=f06d0102041544d89fd381a0478e7923)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.484368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.485979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 153/315] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:45 2022] GET /identity/v3/users/f06d0102041544d89fd381a0478e7923 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:45.491594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:45.589918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:45.610647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4974f-a088-4998-b6da-124cc37d0d5a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.645038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.645690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.645981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.646260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.720643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.722066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.802643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:45.826855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 163/316] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:45.832045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:45.943543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7542bef-d936-4647-914f-82f0cb8f27a4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:45.980455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:45.981753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:45.982242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:45.982718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:45.984380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:45.986143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aded27-3656-42a8-89fa-95fcff782f3c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:45.988641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 154/317] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:45 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:45.996446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:46.095338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60388d-b4d2-4c2e-8119-754f5c921d4e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:46.130435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:46.131109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:46.131400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:46.131678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:46.132554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, scope.project.id=2c52922d9acd4c239eece87719b0285b, user.id=f06d0102041544d89fd381a0478e7923) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:46.132873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:46.134580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab529e-2f55-46b3-8fd9-ff88b7ec67a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:46.142046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 164/318] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:41:45 2022] GET /identity/v3/role_assignments?user.id=f06d0102041544d89fd381a0478e7923&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:47.052328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70849bf-a679-44e9-bdc1-6c7c189190bc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.052844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70849bf-a679-44e9-bdc1-6c7c189190bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.052947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70849bf-a679-44e9-bdc1-6c7c189190bc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.054386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 155/319] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:47.060187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.060586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.060748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.077871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:47.079249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:47.121940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.133490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979c0d-8801-41bd-9d8d-5802676ec0ef None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.133787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979c0d-8801-41bd-9d8d-5802676ec0ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.134066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9979c0d-8801-41bd-9d8d-5802676ec0ef None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.135543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 156/320] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:47.141258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.141550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.141832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.150322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 165/321] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:47.159306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:47.160516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:47.172175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.172923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.241053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 157/322] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:47.263246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.281199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6a264-b9c5-41aa-adbc-6b128d873d4f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.320195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.320913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.321202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.321482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.322653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:47.324911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:47.326525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 166/323] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:47.332084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.373859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d88df55-5d5c-4438-8867-34e1746168b7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.380583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.412096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.412896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.413181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.413456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.424400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:47.432052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:47.433319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:47.434389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 158/324] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:47.438954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.447488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa2e4c-9c5a-4262-bf68-c3dbb758e3af None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.449229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.494464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.495426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.495639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.496074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] PATH_INFO: `/v3/groups` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.496849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:47.498393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:47.517003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 167/325] 10.222.0.24 () {60 vars in 1134 bytes} [Tue May 24 13:41:47 2022] POST /identity/v3/groups => generated 226 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:41:47.568633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b04c9-25d5-425b-b5b3-3db9769add7e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.606009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.606774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.607066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.607347 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.608236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:47.608550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:47.610061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2662a62-0aac-44f9-b65a-f4c0a0a248e8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:47.623850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 159/326] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:47.626394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.649072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c721-387f-4447-b64e-bbefb0267fd7 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.649416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c721-387f-4447-b64e-bbefb0267fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.649726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380c721-387f-4447-b64e-bbefb0267fd7 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.651306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 160/327] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:47.656954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.657299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.657592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.676600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:47.677828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:47.745361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5b438-b82e-4b73-8f0b-9ad60caeab55 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.746108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.749457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:47.755495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 161/328] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:47.779870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.790431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.791887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.791887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.791887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] PATH_INFO: `/v3/users/9b4b07a6b61f4b1498764ee50a189953` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.793784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] RBAC: Authorizing `identity:get_user(user_id=9b4b07a6b61f4b1498764ee50a189953)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:47.795253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:47.797345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 168/329] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/users/9b4b07a6b61f4b1498764ee50a189953 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:47.803519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.909875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd0396-34b2-4996-8833-875eaa029c26 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.936295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8de43-bbf3-4fa1-b554-5724334737a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:47.951514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.952629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.953113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.953574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.966529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:47.974629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:47.975494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:47.976286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:47.978268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:47.978698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 162/330] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:47.979118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:47.979406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:47.979686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] PATH_INFO: `/v3/system/users/9b4b07a6b61f4b1498764ee50a189953/roles/7dd633f5c336427390fa93ce9c2d4053` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:47.981456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9b4b07a6b61f4b1498764ee50a189953, role_id=7dd633f5c336427390fa93ce9c2d4053)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:47.982863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:47.985006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:47.997768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 169/331] 10.222.0.24 () {58 vars in 1334 bytes} [Tue May 24 13:41:47 2022] PUT /identity/v3/system/users/9b4b07a6b61f4b1498764ee50a189953/roles/7dd633f5c336427390fa93ce9c2d4053 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:48.066736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:48.119394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b21b3f-6caf-4b0d-b445-ad311ecdc9ec None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:48.158841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89203-8410-487c-9629-2157428930a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:48.159588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89203-8410-487c-9629-2157428930a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:48.159961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89203-8410-487c-9629-2157428930a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:48.160184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89203-8410-487c-9629-2157428930a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:48.161010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89203-8410-487c-9629-2157428930a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:48.161326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89203-8410-487c-9629-2157428930a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:48.162773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89203-8410-487c-9629-2157428930a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:48.173205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 163/332] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:48.178688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:48.229470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:48.308622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1357a0-3cee-4d26-bc77-30149004c626 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:48.351396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:48.352316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:48.352605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:48.352889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] PATH_INFO: `/v3/users/4ba83c029874425799becae9fe492891` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:48.354117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] RBAC: Authorizing `identity:get_user(user_id=4ba83c029874425799becae9fe492891)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:48.356160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9dee-6440-499d-8fe6-c2dc4f23302f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:48.357789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 170/333] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:48 2022] GET /identity/v3/users/4ba83c029874425799becae9fe492891 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:48.363606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89203-8410-487c-9629-2157428930a8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:48.487743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89203-8410-487c-9629-2157428930a8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:48.508222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:48.523378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:48.524081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:48.524367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:48.524653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:48.525494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] RBAC: Adding query filter params (role.id=b82b050f41064700b9690d91633b2c52, user.id=4ba83c029874425799becae9fe492891, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:48.525808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:48.527795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a4bbc-aaf8-4b5d-ba71-a854be28ef5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:48.540560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 164/334] 10.222.0.24 () {56 vars in 1308 bytes} [Tue May 24 13:41:48 2022] GET /identity/v3/role_assignments?user.id=4ba83c029874425799becae9fe492891&scope.system=all&role.id=b82b050f41064700b9690d91633b2c52 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:50.083530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a74d-e5e5-4948-ac41-d1d857d50573 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.084615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a74d-e5e5-4948-ac41-d1d857d50573 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.085108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4a74d-e5e5-4948-ac41-d1d857d50573 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.086951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 171/335] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:50.093450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486508b9-5389-4846-99a4-1998e0df4b60 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.094307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486508b9-5389-4846-99a4-1998e0df4b60 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.094787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486508b9-5389-4846-99a4-1998e0df4b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.112314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-486508b9-5389-4846-99a4-1998e0df4b60 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:50.113848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-486508b9-5389-4846-99a4-1998e0df4b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:50.195920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 165/336] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:50.216120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4a74d-e5e5-4948-ac41-d1d857d50573 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:50.346898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4a74d-e5e5-4948-ac41-d1d857d50573 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:50.348835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:50.386771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d3cdf-a571-4c17-a23d-f0ea02e0d4da None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.387114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d3cdf-a571-4c17-a23d-f0ea02e0d4da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.387334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d3cdf-a571-4c17-a23d-f0ea02e0d4da None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.389013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 166/337] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:50.393412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:50.394473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.394578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282970-610e-4999-9584-4a165c780afd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.394872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282970-610e-4999-9584-4a165c780afd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.395157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23282970-610e-4999-9584-4a165c780afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.395494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.395999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.407104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:50.416276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:50.417705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:50.418524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:50.421920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 172/338] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:41:50.428383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:50.452244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-23282970-610e-4999-9584-4a165c780afd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:50.453461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-23282970-610e-4999-9584-4a165c780afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:50.563780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 167/339] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:50.607196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23282970-610e-4999-9584-4a165c780afd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:50.650023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81c935-6653-48c4-9a77-202dd13dfaa0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:50.676176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:50.705892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:50.708078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.708568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.709060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.710030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:50.710554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:50.712259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:50.729589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 173/340] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:50.735941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:50.759740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23282970-610e-4999-9584-4a165c780afd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:50.796899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:50.813305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:50.814135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:50.814427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:50.814710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:50.830304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:50.843390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:50.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:50.845565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 168/341] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:50.850446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:50.961884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25caf52e-0a21-416a-9ea1-cad3b58b96cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:50.985815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53e73b-95f5-4fa2-8d70-24848899abe7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:50.996460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.041484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.042198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.042616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.042863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.043645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:51.043989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.045403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.046208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.046943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.047303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.047651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.058440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 169/342] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:51.063735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:51.207461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1bfa48-71b1-49bd-9da4-f57bc4f027f5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:51.231874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.233277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.260649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.261993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.262284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.262569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.270612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.308304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.370647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.372320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.381921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 174/343] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:51.388936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:51.546179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 170/344] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:41:51.550386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:51.573022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feebdde-26aa-4517-8ffc-626fbc5ed51e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:51.624787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.637733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.638604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.638890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.639164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.640622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.642070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bbb9e-95ea-4031-a6cf-374a59023de6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.649141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 175/345] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:51 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:51.657311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36fc5-5ad4-43fe-b9da-ab9ebeffbf53 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.657609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36fc5-5ad4-43fe-b9da-ab9ebeffbf53 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.657890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa36fc5-5ad4-43fe-b9da-ab9ebeffbf53 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.659278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 176/346] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:51.661560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36fc5-5ad4-43fe-b9da-ab9ebeffbf53 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:51.684354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.775982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b30a5-fc7d-4435-9e51-c6c65cef24bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:51.815215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa36fc5-5ad4-43fe-b9da-ab9ebeffbf53 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:51.816852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.817895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.818249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.818597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.819923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.821451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2316a5-f392-4fc8-a124-af7bbba5f538 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.823758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 171/347] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:51 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:51.825179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc16a-0536-4b13-a21d-3a3ec0a255d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.828307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc16a-0536-4b13-a21d-3a3ec0a255d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.828601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddcc16a-0536-4b13-a21d-3a3ec0a255d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.850310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2ddcc16a-0536-4b13-a21d-3a3ec0a255d1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:51.851533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ddcc16a-0536-4b13-a21d-3a3ec0a255d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:51.856301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:51.857164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.857446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.857723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.864385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:51.865841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ff3aa-b63e-43df-aa2d-fb6bd453d16e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:51.871948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 177/348] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:51 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:51.873393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754f-3e1d-4bba-911a-b0035b22e23b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.873682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754f-3e1d-4bba-911a-b0035b22e23b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.873959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a03754f-3e1d-4bba-911a-b0035b22e23b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.875428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 178/349] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:51.878666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03754f-3e1d-4bba-911a-b0035b22e23b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:51.949186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 172/350] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:51.950252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:51.952401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:51.952856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:51.953460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:51.982280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:51.984331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:52.012197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a03754f-3e1d-4bba-911a-b0035b22e23b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.052504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.053347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.053634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.053914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.055093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.056752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:52.057186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.060478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 179/351] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:51 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:52.061436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.077089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 173/352] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:52.078513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.192071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8b5fe-ea4c-45b8-a6e4-3fdf0c819ee8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.205868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72cd5-a152-4380-b1e2-7d0d2dc4a572 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.223259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.239254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.239956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.240249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.240615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.241376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:52.241696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.243175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:52.250298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 180/353] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=b82b050f41064700b9690d91633b2c52 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:52.251068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.259926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.263139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.263139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.263139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.279555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:52.296238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:52.297207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:52.298013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 174/354] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:52.298666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.363354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.385318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032859a1-4d45-4142-b1dc-1afd74201dd1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.424034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffce7e7-aef7-4795-966b-48ae00154850 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.430411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.431090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.431374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.431651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.432632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:52.433602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.435604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04ff09-97fd-4cda-9fbc-fde7abaa431c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:52.442329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 181/355] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:52.443675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439415a-9e86-42ed-a7af-61070fefea0b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.444089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439415a-9e86-42ed-a7af-61070fefea0b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.444368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439415a-9e86-42ed-a7af-61070fefea0b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.446005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 182/356] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:52.446574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439415a-9e86-42ed-a7af-61070fefea0b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.474877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.475692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.476113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.476295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.493997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-4db9a1da-3f83-48f0-a9ba-f145a9fd609e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:52.502452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:52.503428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:52.504783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 175/357] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:52.505451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.505451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.505572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.525631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:52.527414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:52.533580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.590725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439415a-9e86-42ed-a7af-61070fefea0b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.627941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.628799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.629205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.629585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.630529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:52.630908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.632407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:52.643968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 176/358] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:52.644571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 183/359] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:52.644999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.652255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.756187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.790135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acc8551-bd0a-4d2b-bee9-1df1b3f805d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.806412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:52.843046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.843711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.844022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.844299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.871313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1263cd-beb1-4ae9-b74a-380227b37862 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:52.933673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:52.934954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:52.935444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:52.937756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:52.938779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:52.939291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.940934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:52.957099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 177/360] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:52.959935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:52.970535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:52.972510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.166992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.175400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b494021-0463-4699-b2de-4d479daefbd7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.175664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 184/361] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:53.180657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.203768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.233633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.234824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.235320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.235800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.251117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-46ceacaa-efdb-49a0-821a-4318bac12529 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:53.265737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:53.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:53.267900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 178/362] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09b996-0377-46f9-8c6a-885e60a1ec36 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09b996-0377-46f9-8c6a-885e60a1ec36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09b996-0377-46f9-8c6a-885e60a1ec36 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 179/363] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:53.276953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09b996-0377-46f9-8c6a-885e60a1ec36 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.323963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202e41e-f66e-4bbe-8834-c2e4b4fde38e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.372730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.374002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.374480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.374956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] PATH_INFO: `/v3/users/9b4b07a6b61f4b1498764ee50a189953` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.376417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] RBAC: Authorizing `identity:get_user(user_id=9b4b07a6b61f4b1498764ee50a189953)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:53.379536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.382704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 185/364] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/users/9b4b07a6b61f4b1498764ee50a189953 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:53.383977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.408620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09b996-0377-46f9-8c6a-885e60a1ec36 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.450977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.452092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.452485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.452869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.454237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:53.455781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a5b34-3062-4290-9373-11821b992699 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.457728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 180/365] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:53.459110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.459501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.459952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.476863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:53.478313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:53.500588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.518224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.536663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76d505-8bf5-46b8-a92a-f0d25c360b5b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.579062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.579286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 181/366] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:53.579834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.580195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.580480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.581044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.581245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:53.581619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:53.583015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.595913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 186/367] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:53.595913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.706756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505777a-cb64-45cb-ae32-d12c94379ad3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.722584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0b878-6a84-4585-b76a-94b354738d16 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.745646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.746398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.746889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.747370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.749093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] RBAC: Adding query filter params (role.id=7dd633f5c336427390fa93ce9c2d4053, user.id=9b4b07a6b61f4b1498764ee50a189953, scope.system=all) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:53.749650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:53.752262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.762760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.763958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.764462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.764933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.766424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:53.768150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:53.768968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 182/368] 10.222.0.24 () {56 vars in 1308 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/role_assignments?user.id=9b4b07a6b61f4b1498764ee50a189953&scope.system=all&role.id=7dd633f5c336427390fa93ce9c2d4053 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:53.769632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.770934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 187/369] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:53.772081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:53.792041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.794185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:53.902503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada35560-3ca4-49cf-8c38-9d8ff7c358af None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.916280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17d78aa-ee30-4a4f-9b1d-7ab31cc329c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:53.978055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.979255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.980626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.981291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:53.996682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:53.997512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:53.997802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:53.998089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.022716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:54.034158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:54.035044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 188/370] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:54.035801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.084809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.086310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.105498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.166355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af0f59-4856-46e8-b06a-e5abd9568ffc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.171074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.196662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 183/371] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:54.197631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.209597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.210692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.211174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.211747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.212976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=2c52922d9acd4c239eece87719b0285b) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:54.213503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.215779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.222441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 189/372] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=b82b050f41064700b9690d91633b2c52 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:54.223797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.341202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3c0fa-41c6-498f-b772-dc9996bf1bf8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.347295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaeec0-b54d-41c8-9a7e-eaf8ed80786a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.364244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.385217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.386460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.386746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.387025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.388975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:54.388975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.389580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.396625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.397865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.398428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.398929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.400529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.402382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 184/373] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:54.403302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.405615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 190/374] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:54.407588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.410682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.448123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.532550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6ee7f-50e8-4071-906f-469440782407 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.585133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77741914-9d56-4b60-bb94-798f6a08844e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.590047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.590718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.591007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.594066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.650470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.651395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.651713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.652654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.654211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.656914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.658361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.659508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 191/375] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:54.664679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.692894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.694953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.760582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.806223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c44a80-1bcb-4f8e-b189-efe0f2866d3b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:54.834186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 185/376] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:41:54.840194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:54.857353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:54.858010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:54.858304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:54.858581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:54.859437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=2c52922d9acd4c239eece87719b0285b) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:54.859900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:54.861368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:54.868277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 192/377] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:54.943856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:54.970985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e108cb-0aa2-4113-b9df-c47c0078748a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:55.018559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:55.019522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.019832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.020119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] PATH_INFO: `/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.022077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] RBAC: Authorizing `identity:get_project(project_id=4d5cb275b4fb4682b096f6cd5768d3fe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:55.023576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:55.025699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 186/378] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:54 2022] GET /identity/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:55.033011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.180369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8b92b-8fc8-4193-be5a-0db27b525d15 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:55.228682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:55.229636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.229930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.230208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] PATH_INFO: `/v3/groups/73ef6c313c974b408cdd9853f9be6f35` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.236807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:55.240038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] RBAC: Authorizing `identity:get_group(group_id=73ef6c313c974b408cdd9853f9be6f35)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:55.241567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b16358-a0ab-4662-82eb-8d2d308e87b8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:55.243702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 193/379] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/groups/73ef6c313c974b408cdd9853f9be6f35 => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:55.250290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.334429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efca91f-5fc7-4179-80f6-73608dd4cfaa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.334771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efca91f-5fc7-4179-80f6-73608dd4cfaa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.335000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efca91f-5fc7-4179-80f6-73608dd4cfaa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.336561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 194/380] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:55.350500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5867-8fee-4cb4-977d-c4f85753a279 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.351141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5867-8fee-4cb4-977d-c4f85753a279 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.351141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca5867-8fee-4cb4-977d-c4f85753a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.378124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bdca5867-8fee-4cb4-977d-c4f85753a279 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:55.379349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bdca5867-8fee-4cb4-977d-c4f85753a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:55.397953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eea95-ab99-40cd-bcfc-3887bff0def9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:55.440199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:55.441430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.441910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.442380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.443437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] RBAC: Adding query filter params (group.id=73ef6c313c974b408cdd9853f9be6f35, role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=4d5cb275b4fb4682b096f6cd5768d3fe) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:55.443988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:55.445891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:55.453123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 187/381] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/role_assignments?group.id=73ef6c313c974b408cdd9853f9be6f35&scope.project.id=4d5cb275b4fb4682b096f6cd5768d3fe&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:55.532285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 195/382] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:55.568105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.588422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:55.647660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4852e-b044-4876-903e-dbccba7c14cd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.647999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4852e-b044-4876-903e-dbccba7c14cd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.648279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4852e-b044-4876-903e-dbccba7c14cd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.649837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 196/383] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:41:55.656659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.657065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.657231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.686656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:55.696037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:55.706741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78707f6d-cdcd-4ec6-8aed-d0748a4e030a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:55.751139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:55.752357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.758754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.759329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.778993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:55.788172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:55.789047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:55.789809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:55.792876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 188/384] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:55.798931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.804986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 197/385] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:55.849396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.882496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:55.924514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569a152-6192-4762-9edd-9b96b92ae615 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:55.932532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:55.975007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:55.975963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:55.976389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:55.976799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:55.977688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:55.978126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:55.980336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:55.989736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 189/386] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:55.997035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:55.997788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5561d-fe39-423b-9cf9-f2343b7fbc4e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.048851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.049797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.050175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.050526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.072703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:56.085774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:56.086976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:56.087996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:56.090254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 198/387] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:56.095159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.118750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9059-51ef-440e-b3c9-58e61cd83745 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.159156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.169365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.170238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.170529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.170801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.176031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.180407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.183901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 190/388] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:55 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:56.188383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.318417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18a228-a4b0-4170-b4b1-afb50ee7bd37 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.360216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.384221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca0634-9341-4fcb-ad9f-cc4a94d2075f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.388676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.389360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.389649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.389929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.390701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:56.391015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.392449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.409566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 199/389] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:56.416798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.450303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.451914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.452399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.452884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.454484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.456298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.458281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 191/390] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:56.464600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.541466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc353b6-d1d1-4a4a-bd15-dfb2dfec782e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.595578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.601391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.602263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.602551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.602827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.605337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.606780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.617534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 200/391] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:56.620916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.664567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2617b8-c673-4316-a964-f7388f357e91 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.711253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:56.726058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.726952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.727239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.727522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/groups/5a1b0795b5644347b6950e2db07cdf90/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.731243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, group_id=5a1b0795b5644347b6950e2db07cdf90, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.732710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.763271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 192/392] 10.222.0.24 () {58 vars in 1442 bytes} [Tue May 24 13:41:56 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/groups/5a1b0795b5644347b6950e2db07cdf90/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:56.794023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ec2d1-c3a7-402d-ab7f-5c6791229ae9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:56.840626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:56.841516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:56.841807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:56.842086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:56.843323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:56.844873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b3ff6d-c7b3-479d-8920-32f95d4750f4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:56.846629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 201/393] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:56 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:56.852511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:56.978079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50631e2-df0e-4b0e-a768-9691e0ff0c5d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.007931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.017769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.018719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.019330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.019616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/groups/5a1b0795b5644347b6950e2db07cdf90/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.022349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] RBAC: Authorizing `identity:create_grant(project_id=1c3ea61eb18145d2bb5e449fd05afdf7, group_id=5a1b0795b5644347b6950e2db07cdf90, role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.024163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7f138-41a0-4f70-a199-ca9c026a728e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.048045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 193/394] 10.222.0.24 () {58 vars in 1442 bytes} [Tue May 24 13:41:56 2022] PUT /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7/groups/5a1b0795b5644347b6950e2db07cdf90/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:57.244956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172c4a-102f-4287-979a-f9a0887ca1a4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.245480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172c4a-102f-4287-979a-f9a0887ca1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.245653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b172c4a-102f-4287-979a-f9a0887ca1a4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.247609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 202/395] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:57.253171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fcb3-089a-4555-99f2-f18ea7608028 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.253527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fcb3-089a-4555-99f2-f18ea7608028 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.253818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56fcb3-089a-4555-99f2-f18ea7608028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.277243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7f56fcb3-089a-4555-99f2-f18ea7608028 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.278484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7f56fcb3-089a-4555-99f2-f18ea7608028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.363544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 194/396] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:57.391925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b172c4a-102f-4287-979a-f9a0887ca1a4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.545061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b172c4a-102f-4287-979a-f9a0887ca1a4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.568537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.573097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96959-0010-4d98-b888-068560da5e2c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.573882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96959-0010-4d98-b888-068560da5e2c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.574219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96959-0010-4d98-b888-068560da5e2c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.578537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 195/397] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:57.583208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.583506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.583788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.593700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.594500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.594784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.595065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.611695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:57.616844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:57.618046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:57.619454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:57.620524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:57.621505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 203/398] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:57.626695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.748518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cfc291-b439-4cd1-8d71-d37155c09295 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:57.770223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 196/399] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:41:57.789403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:57.790077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:57.790360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:57.790639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:57.791401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:57.791711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:57.793709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:57.804503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 204/400] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:57.807103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.810997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:57.890999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.913540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:57.992401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286d4073-5706-4d44-994b-49d17aa4474e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.008014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0789fb-bbbc-4f7d-8634-ed81a52488dd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.050428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.051518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.051834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.052124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.057683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.058967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.059420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.059961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.061393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.062991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.063614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:58.065338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 205/401] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:58.076715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:58.077789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:41:58.078713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 197/402] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:41:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:58.080561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.086601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.200976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9451a-59b1-4e94-b9ef-1dc3b6a904c7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.223526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.241032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.241965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.242263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.242599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.243371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:58.243763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.246429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.257353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 198/403] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:58.264970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.346700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acdc660-1b44-4ff6-a7eb-185fa5172f95 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.363146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.391207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e1f74-181b-453b-83c3-12f4069bfbde None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.424987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.425870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.426160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.426441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.431000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.433023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.444934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 206/404] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:58.450346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.467773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.469010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.469500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.469999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.472418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.474085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56cb7b6-57f6-4c67-90cc-06f72e119265 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.476229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 199/405] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:58.477627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9192-7ffc-4942-912f-e03d8d5d5251 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.478149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9192-7ffc-4942-912f-e03d8d5d5251 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.478630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e9192-7ffc-4942-912f-e03d8d5d5251 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.480601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 200/406] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:58.481520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e9192-7ffc-4942-912f-e03d8d5d5251 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.543523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.580564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a570e9-dc8d-4fce-a73d-f008af787dd9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.608560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e9192-7ffc-4942-912f-e03d8d5d5251 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.617019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.617916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.618205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.618484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/groups/5a1b0795b5644347b6950e2db07cdf90/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.621314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, group_id=5a1b0795b5644347b6950e2db07cdf90, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.622869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36822fa5-1d6b-4d60-8a1f-9c07e87345c6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.641453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 207/407] 10.222.0.24 () {58 vars in 1442 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/groups/5a1b0795b5644347b6950e2db07cdf90/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:41:58.642851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.643137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.643426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.657946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.659266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.659766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.660310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.663177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:58.664352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.664464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:41:58.666182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.669213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 201/408] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:58.675590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.702474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.748507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 208/409] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:41:58.775905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:58.899898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6ddc1-acfa-4398-a0b1-360a0c1e4a1e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.916394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:58.928098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128028d-ec2b-4446-a842-d3b728f0ab2e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:58.953038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.953927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.954215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.954496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b/groups/5a1b0795b5644347b6950e2db07cdf90/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.957863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c52922d9acd4c239eece87719b0285b, group_id=5a1b0795b5644347b6950e2db07cdf90, role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:58.959309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:58.969895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:58.970994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:58.971426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:58.971878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:58.984119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:41:58.986649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 202/410] 10.222.0.24 () {58 vars in 1442 bytes} [Tue May 24 13:41:58 2022] PUT /identity/v3/projects/2c52922d9acd4c239eece87719b0285b/groups/5a1b0795b5644347b6950e2db07cdf90/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:41:58.993557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:41:58.994690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:41:58.995761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:41:58.997605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 209/411] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:41:59.003970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.113866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cc6a-5a53-4678-9c8f-410301a3d9b2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.158120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.159144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.159634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.160132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.161096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:41:59.161645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.163206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.173487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 203/412] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:41:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:59.178735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.217673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:59.313614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf748d-df61-458a-94a3-f986127d9cdc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.365179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.366067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.366352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.366637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] PATH_INFO: `/v3/groups/73ef6c313c974b408cdd9853f9be6f35` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.367887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] RBAC: Authorizing `identity:get_group(group_id=73ef6c313c974b408cdd9853f9be6f35)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.369351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.372339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 210/413] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:41:59 2022] GET /identity/v3/groups/73ef6c313c974b408cdd9853f9be6f35 => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:41:59.377555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.431387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:59.544077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416237cb-9d6e-481f-aff7-c3fa332a0e49 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.590087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.591019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.591312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.591594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] PATH_INFO: `/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.596617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] RBAC: Authorizing `identity:get_project(project_id=4d5cb275b4fb4682b096f6cd5768d3fe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.598083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f9b59-fbc2-4a8c-aa39-0d799a332672 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.600324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 204/414] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:41:59 2022] GET /identity/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe => generated 344 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:41:59.607808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:41:59.753154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:41:59.764951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13143180-b8b6-4de5-86c2-5ff435b843c2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:41:59.808177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:41:59.809069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.809353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.809634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] PATH_INFO: `/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe/groups/73ef6c313c974b408cdd9853f9be6f35/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.813150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d5cb275b4fb4682b096f6cd5768d3fe, group_id=73ef6c313c974b408cdd9853f9be6f35, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:41:59.814690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b41636-5627-4521-8955-e174cd5b0dd4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:41:59.840022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 211/415] 10.222.0.24 () {58 vars in 1442 bytes} [Tue May 24 13:41:59 2022] PUT /identity/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe/groups/73ef6c313c974b408cdd9853f9be6f35/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:41:59.923968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392bb3-ee6e-475a-ad25-7370b0ced8d0 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.924995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392bb3-ee6e-475a-ad25-7370b0ced8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.925531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e392bb3-ee6e-475a-ad25-7370b0ced8d0 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.927279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 205/416] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:41:59.933973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30508cb-03a5-4d99-8a2b-53eefedb7ef3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:41:59.934630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30508cb-03a5-4d99-8a2b-53eefedb7ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:41:59.935107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30508cb-03a5-4d99-8a2b-53eefedb7ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:41:59.954456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a30508cb-03a5-4d99-8a2b-53eefedb7ef3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:41:59.956056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a30508cb-03a5-4d99-8a2b-53eefedb7ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:00.040654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 212/417] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:00.065155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392bb3-ee6e-475a-ad25-7370b0ced8d0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.171362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567137c6-f505-4d7c-8c01-887f585a5dda None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.172044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567137c6-f505-4d7c-8c01-887f585a5dda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.172719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567137c6-f505-4d7c-8c01-887f585a5dda None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.174546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 213/418] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:00.180559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.181103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.181780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.190180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e392bb3-ee6e-475a-ad25-7370b0ced8d0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.194780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.202876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:00.205187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:00.255105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.256340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.256850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.257395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.273623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:00.287876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:00.288841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:42:00.289628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:00.293563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 206/419] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:00.300511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.349500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 214/420] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:00.393212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.418681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.473873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4aaa9-7204-4693-9754-b8c6a7553b3f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.525042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.525730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.527626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.531978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.532861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:00.533191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.534620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.558343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 207/421] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:00.565420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11614e86-c8d3-47df-a576-71f17795f68e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.568779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.606016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.607077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.607622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.608081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.612571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.619374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:00.629359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:00.630259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:00.631051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 215/422] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:00.637414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.740645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:00.772637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd8068-1616-4658-8f62-46c59ec30f7b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.783071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e6259-3906-4408-8174-68d2bb1e3177 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.826761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.828570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.829758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.830574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.832579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:00.834458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.837475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.853802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:00.855232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:00.857350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 216/423] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:00.857883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:00.858519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:00.863076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:00.950974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:00.959627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:00.987843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925089c-ac7a-4363-9ee8-5f9e38eb9a2c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:00.993593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.033076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.033747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.034042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.034324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.040132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.141362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 208/424] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:01.145295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.165851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.168125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.277191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 217/425] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:01.284828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.355503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.365055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a98c8f-7e3e-4b1e-8044-7782039b4a66 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.386916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.405120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.405913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.406207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.406485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.407693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.409285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.411320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 209/426] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:01.416919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.476691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e270df17-6cab-446d-af26-ace9bf935bc2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.520994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.521955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.522311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.522655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] PATH_INFO: `/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.523995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] RBAC: Authorizing `identity:get_project(project_id=1c3ea61eb18145d2bb5e449fd05afdf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.526481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d895d7-2b90-4c94-b7c9-cfc112f32221 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.529080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 218/427] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/projects/1c3ea61eb18145d2bb5e449fd05afdf7 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:01.530446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd65-5337-4e05-bc03-34c292105006 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.530813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd65-5337-4e05-bc03-34c292105006 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.531154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7fd65-5337-4e05-bc03-34c292105006 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.532751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 219/428] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:01.533897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7fd65-5337-4e05-bc03-34c292105006 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.566221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199da645-2bd0-4d56-a2f8-ca0bccd229ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.611719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.613268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.613562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.613848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.615095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.617498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec9fb5-7ced-49a1-b56b-2a74decc3ed1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.620565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 210/429] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:01.622725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.623016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.623345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.646162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:01.647712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:01.657824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7fd65-5337-4e05-bc03-34c292105006 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.661559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.703252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.704313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.704712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.705087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.706705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.708864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.715693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 220/430] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:01.715693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.719701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.750934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 211/431] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:01.750934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.844905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb97e4e-3950-415b-a67a-98ea2197bde9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.878657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618d2c7-fe9d-422b-9ca4-102e0015d8c7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:01.885104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.886304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.886590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.886875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.887715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:01.888163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.890169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.898857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 221/432] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=b82b050f41064700b9690d91633b2c52 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:01.899926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:01.934543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:01.936115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:01.936668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:01.937206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:01.938719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:01.939278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:01.948776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:01.951257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:01.959191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 212/433] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=1c3ea61eb18145d2bb5e449fd05afdf7&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 611 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:02.032884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0158fa72-5ebf-4bb5-b01e-334bd59f287a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.073402 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.074391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.074688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.074969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.087661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-11c78085-3640-4b5b-8805-edc5af8c24ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:02.096467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:02.097416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:02.098213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 222/434] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:02.104362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.135003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258f528-4457-4275-9f48-d7bd52237828 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.135388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258f528-4457-4275-9f48-d7bd52237828 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.135648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6258f528-4457-4275-9f48-d7bd52237828 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.141411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 223/435] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:02.144050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.144348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.144630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.171936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:02.173165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.173558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.245266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad420c5-c026-424e-9b87-defbd6e131fd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.288626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 224/436] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:02.291290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.292914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.292914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.292914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.293569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] RBAC: Adding query filter params (name=member) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:02.293948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.295399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.308179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 213/437] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:02.313854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.322337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.352553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.452676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9796385-3449-4d1e-adc5-eaec38d5a375 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.455063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103379a8-fc4f-4bdb-abb3-e2a1d7a37c32 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.467253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.501095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.502284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.502585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.502868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.513872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.514709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:02.515088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.515581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.516109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:02.522884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:02.523779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:02.524628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:02.525696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 214/438] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:02.530783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.603037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.603997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd968ccd-994f-4f04-ba0e-7e800d8b6998 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.633614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.658287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f15dc3-54c1-4836-bf45-3418c9d0182c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.697749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.698948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.699426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.699709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.700990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:02.700990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.702210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b9029-4e6b-430b-aa2c-b75e190e8c7f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.710742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 215/439] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:02.712873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aee14-4838-4a8a-a178-1724865f684b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.713167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aee14-4838-4a8a-a178-1724865f684b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.713485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504aee14-4838-4a8a-a178-1724865f684b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.715593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 216/440] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:02.718795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aee14-4838-4a8a-a178-1724865f684b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.729131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 225/441] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:02.735103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.735581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.735889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.751701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.752603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:02.753760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:02.822863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504aee14-4838-4a8a-a178-1724865f684b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.823445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 226/442] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:02.824340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:02.857969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.858615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.858900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.859180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.880932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.923713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6223d-f9e8-499c-8b17-6433874eb641 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:02.926048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.927586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.957826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:02.958068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:02.958979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:02.959321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:02.959552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:02.960764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:02.962190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:02.964439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 227/443] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:02.965076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.028841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 217/444] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:03.029602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.066978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521eafa-2d8d-419a-b56c-829d12ae715d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.090926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.102780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.103573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.103896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.104278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.114356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:03.122588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:03.123495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:03.124341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 228/445] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:03.124988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.136894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd519da-154b-46f3-919f-8b5b09b97a6f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.165719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.172253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.173108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.173394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.173889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.174843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.176308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.177916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 218/446] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.178675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.223143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9ff859-c747-46ff-a60e-9dec7c22ae46 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.257881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.258994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.259280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.259559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] PATH_INFO: `/v3/projects/2c52922d9acd4c239eece87719b0285b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.261392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2c52922d9acd4c239eece87719b0285b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.262863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.264601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 229/447] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/projects/2c52922d9acd4c239eece87719b0285b => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.265364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.281272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585dbd9-aa6c-480f-88ed-e1330568ef7c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.314933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.315599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.315920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.316207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.316962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:03.317274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.318643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.319323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.326876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 219/448] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.327666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.368611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147327d7-781c-42be-ad21-c400ada501b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.389685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.403315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.404003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.404292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.404577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.405444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=b82b050f41064700b9690d91633b2c52, scope.project.id=2c52922d9acd4c239eece87719b0285b) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:03.405762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.407203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.413111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 230/449] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=b82b050f41064700b9690d91633b2c52 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.413806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.433168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8293361-4742-49d6-9e27-c1db547125ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.473396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.474282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.475008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.475008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] PATH_INFO: `/v3/groups/5a1b0795b5644347b6950e2db07cdf90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.476294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] RBAC: Authorizing `identity:get_group(group_id=5a1b0795b5644347b6950e2db07cdf90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.477883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.484982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 220/450] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/groups/5a1b0795b5644347b6950e2db07cdf90 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.485945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.542902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8426c-e958-4746-80aa-61b0676355e4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.561465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.582203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.582876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.583167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.583452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.605701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd9686-8e7c-4250-af65-a81090833ae0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.638251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:03.645793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.646454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.646745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.647038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.647899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] RBAC: Adding query filter params (group.id=5a1b0795b5644347b6950e2db07cdf90, role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=2c52922d9acd4c239eece87719b0285b) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:03.648580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.649668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.657209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 221/451] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/role_assignments?group.id=5a1b0795b5644347b6950e2db07cdf90&scope.project.id=2c52922d9acd4c239eece87719b0285b&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.670026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.671542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.788132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 231/452] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:03.794943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.909217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4dfe3e-f9bb-4174-8e0e-5dec371703eb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:03.944058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:03.944914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:03.945202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:03.945475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] PATH_INFO: `/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:03.946662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] RBAC: Authorizing `identity:get_project(project_id=4d5cb275b4fb4682b096f6cd5768d3fe)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:03.948301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:03.949978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 222/453] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/projects/4d5cb275b4fb4682b096f6cd5768d3fe => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:03.955568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:03.973612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:04.059779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fd034-835c-48fe-9b1c-93ba32667380 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:04.091749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:04.092618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.092902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.093179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] PATH_INFO: `/v3/groups/73ef6c313c974b408cdd9853f9be6f35` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.094327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] RBAC: Authorizing `identity:get_group(group_id=73ef6c313c974b408cdd9853f9be6f35)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.095761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2043d-7c6a-46a5-a3c9-b628dc828aff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.097386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 232/454] 10.222.0.24 () {56 vars in 1180 bytes} [Tue May 24 13:42:03 2022] GET /identity/v3/groups/73ef6c313c974b408cdd9853f9be6f35 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:04.103204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:04.162728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:04.200195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c579401-dab4-461f-8a98-65c26b9be849 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:04.231900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:04.232573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:04.232839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:04.233122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:04.233980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] RBAC: Adding query filter params (group.id=73ef6c313c974b408cdd9853f9be6f35, role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=4d5cb275b4fb4682b096f6cd5768d3fe) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:04.234271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:04.235692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659fef-f3c6-4c66-a14f-cb246e1619a8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:04.241377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 223/455] 10.222.0.24 () {56 vars in 1376 bytes} [Tue May 24 13:42:04 2022] GET /identity/v3/role_assignments?group.id=73ef6c313c974b408cdd9853f9be6f35&scope.project.id=4d5cb275b4fb4682b096f6cd5768d3fe&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:06.907597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574288a0-e69a-42fc-8853-ec701e7be187 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.908617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574288a0-e69a-42fc-8853-ec701e7be187 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.909115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574288a0-e69a-42fc-8853-ec701e7be187 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.910705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 233/456] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:06.915971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6937-9b03-4349-a551-cae531afea1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:06.917418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6937-9b03-4349-a551-cae531afea1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:06.917907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e6937-9b03-4349-a551-cae531afea1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:06.937918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-774e6937-9b03-4349-a551-cae531afea1b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:06.939444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-774e6937-9b03-4349-a551-cae531afea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:07.035680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 224/457] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:07.036171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fafbb-ce8e-44bf-a164-fdcc04a35b4e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.036722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fafbb-ce8e-44bf-a164-fdcc04a35b4e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.037211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796fafbb-ce8e-44bf-a164-fdcc04a35b4e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.038985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 234/458] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:07.045675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c84f5-e564-4817-8566-fb41b83c0aac None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.045973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c84f5-e564-4817-8566-fb41b83c0aac None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.046257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c84f5-e564-4817-8566-fb41b83c0aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.065020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fafbb-ce8e-44bf-a164-fdcc04a35b4e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.069087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3e1c84f5-e564-4817-8566-fb41b83c0aac None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:07.070276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3e1c84f5-e564-4817-8566-fb41b83c0aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:07.151387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.184345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 225/459] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 24 13:42:07.185797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe259d7-1233-42e0-a45e-0255ee5e01dd None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.186198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe259d7-1233-42e0-a45e-0255ee5e01dd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.186488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe259d7-1233-42e0-a45e-0255ee5e01dd None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.192225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 226/460] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:07.194090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.194388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.194670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.214458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:07.216074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:07.225491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.243846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796fafbb-ce8e-44bf-a164-fdcc04a35b4e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.281481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.282552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.282997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.283416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.296308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.298098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.303936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 227/461] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:07.305027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:07.318968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:07.320167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:07.321009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:07.321009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:07.321009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.321009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:07.323114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 235/462] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:07.328912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.421635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62fec6-98d2-46d6-81c1-be2ebcce7f6a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.439734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.455420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e3192-b88a-4d60-8af4-1ce4ef5ac06c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.459192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.460167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.460527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.460881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.472801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.474409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:07.493972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:07.495339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:07.496302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:07.496302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:07.496302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.496302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:07.497664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.498341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 228/463] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:07.498867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.499282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.499546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.500056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.500988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:07.501508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.503936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.508687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.530449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 236/464] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:07.531766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.610193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76b899-3621-467e-b5fd-7b68a959a313 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.654168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ab71e-07ab-422c-b3bc-d954968a96a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.656834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.657938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.657938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.658214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.670801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.672368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:07.695055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:07.695940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:07.696756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:07.696756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:07.696756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:07.696756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:07.696756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 229/465] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:07.698003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.698745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.699629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.700143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.700611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.701789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:07.703254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.704892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.725826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 237/466] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:07.726973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.727296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.796932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:07.821522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7a7f-da0e-4617-8cc7-50b52563c91e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.845513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe265f-385c-40d1-a84d-5d08342c1259 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:07.857597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.858263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.858547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.858823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.859534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.860989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.881288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 230/467] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:07.882158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:07.894840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:07.895935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.896299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.896663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.897505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:07.897973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:07.899627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5d059-50a7-4b44-a612-f8071f8fd54f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:07.914421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 238/468] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:07.915971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c5ae-1051-4568-bd08-66555d20156b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:07.916389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c5ae-1051-4568-bd08-66555d20156b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:07.916924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c5ae-1051-4568-bd08-66555d20156b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:07.922525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 239/469] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:07.923482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c5ae-1051-4568-bd08-66555d20156b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.015340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.043201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8a3c2-4470-4772-a568-c2b416100bd3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.071242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c5ae-1051-4568-bd08-66555d20156b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.086895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.087474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.087860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.088907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.089726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.091344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906d5c2d-1b99-4f4f-a417-ce577cbf89e2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.095619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.113871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 231/470] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:08.115156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552008-38a0-492d-ac6e-693e661ba57c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.115455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552008-38a0-492d-ac6e-693e661ba57c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.115744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552008-38a0-492d-ac6e-693e661ba57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.116560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.117452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.117911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.118359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.119250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.121639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.137358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-49552008-38a0-492d-ac6e-693e661ba57c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:08.138810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-49552008-38a0-492d-ac6e-693e661ba57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:08.146288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 240/471] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:07 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:08.236805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 232/472] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:08.266568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.377781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d32c41-785a-4e3a-b5a9-02c29aaf5f31 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.378319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d32c41-785a-4e3a-b5a9-02c29aaf5f31 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.378609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d32c41-785a-4e3a-b5a9-02c29aaf5f31 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.381770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 233/473] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:08.389786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.390377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.390663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19b49e-de65-468c-bfbd-a3a25e322678 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.391086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.414604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:08.416115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:08.438632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.439438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.439723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.440032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.459947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.461434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:08.480116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:08.481088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:08.481927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:08.481927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:08.481927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:08.481927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:08.481927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 241/474] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:08.487603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.498970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.517438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 234/475] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:08.536244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.545047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.666854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d39f84-74c4-40f0-b6df-806d6caea0c9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.724054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab7638-c532-4241-8160-05fa89b54e73 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.726949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.727641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.727966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.728252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.729529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:08.729848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.731388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.756945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 242/476] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:08.762332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.783436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:08.784486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:08.785039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:08.785525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:08.786546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:08.789434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:08.807860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 13:42:08.807860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 13:42:08.807860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [parameters: {'id': '87b6703aff9a4152b66b807e5231edd1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 13:42:08.807860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:42:08.832398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return ref.to_dict() May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:42:08.833373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 13:42:08.834224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 13:42:08.835062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:42:08.836675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:42:08.838729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': '87b6703aff9a4152b66b807e5231edd1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:08.839503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 13:42:08.840341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:08.843694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 235/477] 10.222.0.24 () {60 vars in 1131 bytes} [Tue May 24 13:42:08 2022] POST /identity/v3/roles => generated 144 bytes in 299 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:08.850044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:08.933218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:08.956145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f124a34-db9b-47c7-a61b-18a40844fe26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:08.966837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:09.028641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.029321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.029607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.029881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.030600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.032066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.041954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cecdc69-05fa-40c1-aabe-495b9f4b0e11 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.083599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.084523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.084814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.085093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.086699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 243/478] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:08 2022] POST /identity/v3/users => generated 249 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:09.098161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-695ca0c9-59c6-45f6-844e-5e93badcac6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:09.106892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:09.107789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:09.108613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 236/479] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:09.117254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:09.294171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f4e5-3c75-4af8-9e25-4b8ed2252798 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:09.329791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:09.330474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:09.330764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:09.331043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:09.331827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:09.332147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:09.333534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754f22-184d-4a17-8022-51a3a32fd6d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:09.343995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 244/480] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:10.715507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a43b-820d-423d-8f90-63319b00edaf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.715507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a43b-820d-423d-8f90-63319b00edaf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.715507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4a43b-820d-423d-8f90-63319b00edaf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.715507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 237/481] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:10.722310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b27599-d654-40c5-83b5-ea110a56b746 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.722310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b27599-d654-40c5-83b5-ea110a56b746 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.722310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b27599-d654-40c5-83b5-ea110a56b746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.727191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:10.745007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-34b27599-d654-40c5-83b5-ea110a56b746 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:10.747040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34b27599-d654-40c5-83b5-ea110a56b746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:10.842640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 245/482] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:10.876568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4a43b-820d-423d-8f90-63319b00edaf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:10.983723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f18f-3c37-4083-aae8-f4eea6f8dd82 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.984049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f18f-3c37-4083-aae8-f4eea6f8dd82 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.984336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05f18f-3c37-4083-aae8-f4eea6f8dd82 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.985714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 246/483] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:10.996924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57390b94-f87b-4805-8500-385c035a22d7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:10.997211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57390b94-f87b-4805-8500-385c035a22d7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:10.997493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57390b94-f87b-4805-8500-385c035a22d7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:10.999089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 247/484] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:11.002838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.003128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.003406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.024404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:11.027414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:11.029318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4a43b-820d-423d-8f90-63319b00edaf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.048775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.067589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.068554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.068947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.069351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.080361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f86389ac-d325-48b4-b70e-f415d2161af8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:11.088955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:11.090087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.091080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:11.095902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 238/485] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:11.095902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.095902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.095902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.103561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 248/486] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:11.104638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.116706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:11.118246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:11.171499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.193385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 239/487] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:11.194519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.215370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f58a3a-d652-4025-bc3b-f97a9975c0e1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.253707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.254429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.254818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.255074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.255845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:11.256169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.257771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.266796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 249/488] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:11.267662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.300393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ecff2-e05a-48b9-890e-2163b79c628b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.309585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.337400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.338885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.339449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.339875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.350656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:11.359043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:11.360185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.361188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:11.363080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 240/489] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:11.363926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.382397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339977b1-a8be-47a1-bc87-b300f4e35659 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.408363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.422237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.423355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.423924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.424527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.436389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:11.444769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:11.445624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:11.446427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:11.449963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 250/490] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:11.451284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.470228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952bec9-6c64-4239-b36c-57715b40828e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.506384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.507194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.507485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.507772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.520629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.522131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:11.541513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:11.542656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 241/491] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:11.544082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.566232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.577921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70350b-96ae-4654-98d7-92f2af46e2e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.615432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.616487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.616977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.617468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.618424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:11.619108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.621135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.630692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 251/492] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:11.631989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.656880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552d4f2-1573-4e16-987f-ff235fcec2fd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.691492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.702256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.703392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.704090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.704551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.706048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:11.706559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.708986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.721070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 242/493] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:11.722594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.757365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508fdff-68ff-4a3a-87b0-42bfe751dbab None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.803046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.804039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.804404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.804748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.805973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:11.806295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.808569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.823881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 252/494] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:11.824475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:11.824721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.858276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2d61a-1b07-4504-98dd-e16213b1d796 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.914615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:11.915345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:11.916735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:11.917213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:11.931520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:11.935298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:11.954430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902ef4b3-ab29-4809-9e95-40110b23e6f1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:11.956877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:11.957706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:11.958964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:11.958964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:11.958964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:11.958964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:11.958964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 243/495] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:11.959377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:11.966804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.006846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.007706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.008277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.008559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.023088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.025442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:12.046574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:12.047468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:12.048317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:12.048317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:12.048317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:12.048317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:12.050310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 253/496] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:12.051502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.082663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb4df6-143c-45b3-8e37-04a45e8699da None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.110941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.116226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.117018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.117303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.117583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.125785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.127211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:12.143274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:12.144212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:12.145138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 244/497] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:12.145689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.157762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf24b-921d-42f1-9e60-d44649a2c49e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.191489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.192104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.192387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.192660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.193373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:12.193678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.195050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749df0b9-b364-44f5-bb1a-ebfa37286c28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.206735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 254/498] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:12.208083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf8ef-affb-4557-85a0-a91f69a90920 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.208378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf8ef-affb-4557-85a0-a91f69a90920 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.208664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddf8ef-affb-4557-85a0-a91f69a90920 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.209908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 255/499] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:12.211415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e319b1-0324-4f55-9b26-f47085a62857 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.211778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e319b1-0324-4f55-9b26-f47085a62857 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.212091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e319b1-0324-4f55-9b26-f47085a62857 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.214161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 256/500] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:12.214825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e319b1-0324-4f55-9b26-f47085a62857 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.237861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.251394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c51b9b-7a82-4b5f-8971-1e2cc40e08d2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.285589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.286249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.286552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.286813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.287536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:12.287871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.289251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.301412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 245/501] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:12.302217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.319984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e319b1-0324-4f55-9b26-f47085a62857 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.352124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.352785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.353067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.353342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.354828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:12.355140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.356000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.356600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72beb9ad-d3d3-498c-9b69-ddcc6bdd041c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.394241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 257/502] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:12.395494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.395783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.396093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.408169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675c34d-1f5a-488a-965c-a4a48ae7d99e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.412432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:12.413618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:12.443175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.443927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.444218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.444496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.453008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.454448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0a34ad5b-042b-4262-aeb1-dac3b00c7277 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:12.469587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:12.470461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:12.471312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 246/503] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:12.472322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.472585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.472867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.483640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 258/504] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:12.484740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.488709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:12.489913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:12.498763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.561916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 247/505] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:12.562768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.592209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1f7c1-0df4-49de-971d-89e049a12d56 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.610598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.626335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.627135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.627422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.627698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.634918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.636420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:12.652757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:12.653647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:12.654510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 259/506] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:12.655067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.671462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ebc61-43c8-4a80-b8bf-0413a8a8ac04 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.705050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.705624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.705907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.706183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.719299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.756445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f5b0a-260c-4913-9033-ff15caeab0b6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.773430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.774886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.789941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.790606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.790895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.791298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.792173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:12.792486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.793921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.811107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 260/507] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:12.811924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.817185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.875231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 248/508] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:12.876194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:12.913598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7364d-3461-4ed0-b4d1-280adfa55580 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.936575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:12.947880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:12.948661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:12.948944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:12.949223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:12.959312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:12.960734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:12.972651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641a243-78d5-4561-a921-068f4c87ad8a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:12.977831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:12.979277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:12.980142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:12.980142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:12.980142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:12.980142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:12.980142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 261/509] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:12.980518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.006877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.007667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.008248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.008529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.019060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:13.026889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:13.027753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 249/510] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:13.029313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.033052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.082133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2402ed-940f-42c0-948a-8172922d9349 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.115046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.115707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.116022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.116302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.117138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:13.117448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.118879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.128882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1ae8e-61c7-4636-b8c8-a9a575de5f6e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.136162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 262/511] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:13.137153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.163294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.163966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.164307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.164595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.171152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.230285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.231680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.237562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f8b7d9-a0e5-4f29-914b-656ddbdb2cbf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.238024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.273332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.274120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.274408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.274689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.286198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.287653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:13.305054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:13.305909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:13.306971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:13.306971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:13.306971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.306971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:13.306971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 263/512] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:13.307292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.343253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 250/513] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:13.343253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.393595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.409332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7ddc-f725-4f16-b59b-0fe9c82ca9ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.437002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81cbd3-1e57-4251-b8e1-c00681625e70 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.438503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.444174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.445241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.445528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.445807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.446537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:13.447240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.449130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.460594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 264/514] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:13.461347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.471842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.472511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.472799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.473078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.473833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:13.474141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.475502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.483726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 251/515] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:13.484482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.568065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b8a6e7-8ee5-47ba-820b-795f3af5821a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.584728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c365d425-65c1-4473-8ad0-9b1d9b1635a4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.605407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.606068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.606355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.606636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.618849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.620432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.620719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.620995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.621723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:13.622039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.622305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.623437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.635857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 252/516] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:13.636656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.661514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.677193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.678665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.735312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575c25b-d1ac-40cc-8f0b-a88b8926c2ef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.770099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.770884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.771177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.771455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.776192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 265/517] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:13.776995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.781627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.783115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:13.801622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:13.802642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:13.803665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:13.803665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:13.803665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.803665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:13.805388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 253/518] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:13.807292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.824694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.878947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5ae12-dca0-4fd1-9e23-a68106dfa39a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.884314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:13.908073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7ec03-2d49-4d7c-bfbd-5fe0c9f54eec None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:13.912891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.913565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.913855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.914138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.914852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.916872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.937605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 266/519] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:13 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:13.938591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:13.945580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:13.946398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:13.946684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:13.946962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:13.961418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:13.963306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:13.983924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:13.984856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 254/520] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:13.990631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.069841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d8dd3-f735-4fd3-94ac-882e980f6c02 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.071691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.102976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc899-4a5f-4a81-8a44-88e83d969280 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.111791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.112869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.113315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.113597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.121537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.122884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.136429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:14.139391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:14.140488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:14.141430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 267/521] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:14.142223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.148903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.149903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.150192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.151423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.164227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.165705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:14.184870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:14.185770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:14.186606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:14.186606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:14.186606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:14.186606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:14.186606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 255/522] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:14.187028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.259557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1759f9a-d777-4ae0-9057-41403fd034af None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.292792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13227ecb-8453-4b98-aff5-9118499a06c7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.296038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.296703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.296989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.297269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.297997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:14.298307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.299684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.313283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 268/523] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:14.314103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.331429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.332128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.332418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.332700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.333435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:14.333748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.335157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.340649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.347851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 256/524] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:14.348272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.390501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.429672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da0424-820a-42d9-9043-d6216fe41fe5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.453155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189538c5-c50c-4b4b-ad15-7b3128f94dae None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.464974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.465640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.465929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.466212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.467009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:14.467324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.468799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.489972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 269/525] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:14.490141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.490677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.490813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.491027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.491243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.492001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:14.492317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.493700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.516573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 257/526] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:14.516573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.587462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.610858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483a5a3-cdc9-4a62-b2bc-d6ce4df1a9b0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.615196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.618487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bc66d-5295-4907-ab2d-4a7564502f5c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.647782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.648962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.649252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.649532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.656664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.656817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.657526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.657814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.658090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.658271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.665336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.667356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:14.672866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:14.673679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:14.674492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 270/527] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:14.674936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:14.687197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:14.688204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:14.689062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:14.690069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 258/528] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:14.690954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.785767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b799b85-373d-40e2-925c-d65fbd5b11da None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.796105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8b47d-43d7-41f9-b58f-e5b275f67ef0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:14.822307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.823128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.823427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.823712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.824599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:14.824919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.826369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.839845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 271/529] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:14.840090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:14.840202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.840288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:14.840288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:14.840288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:14.849499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:14.850001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.851858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:14.877260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:14.878194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:14.879011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 259/530] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:14.879467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:14.920466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:14.980784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3394b7b9-aa00-49e0-8214-9a5f4fd6c493 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.016981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b6d81-f523-49da-af69-7a419eaeda8f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.032197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.032803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.033100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.033385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.034193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:15.035186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.036858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.051686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 272/531] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:15.052595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:15.066601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.067878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.068833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.069350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.070671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:15.071410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.073806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.118287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 260/532] 10.222.0.24 () {56 vars in 1139 bytes} [Tue May 24 13:42:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:15.119919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:15.135584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:15.182663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6afa5-bb4e-4717-b5fb-ad9e1fb2ad7f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.212573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:15.233737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.235068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.235552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.236192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.237931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:15.238617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.240331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469d575-8df6-44c1-aee8-23d3ea22b3ca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.240754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.261492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 273/533] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:15.263240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:15.285962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.287206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.287980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.288484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.289523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=9da57903126d421b8ac6c0e26b3fb3b2) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:15.290053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.292462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.306278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 261/534] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:15 2022] GET /identity/v3/role_assignments?user.id=9da57903126d421b8ac6c0e26b3fb3b2&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:15.306278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:15.461611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9ecd6-7cb8-4994-9ef0-d797bde7d204 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.503011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de4fba-cf0d-4e2e-8982-0d4979163aec None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.516228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:15.524428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.525918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.526433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.526960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.568243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.568243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.568243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.568243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:15.568609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=eb86cbb65b734fc69bf7604414f8c9cf) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:15.568609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.570395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.579117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 262/535] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:15 2022] GET /identity/v3/role_assignments?user.id=eb86cbb65b734fc69bf7604414f8c9cf&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:15.660110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:15.661864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:15.815377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 274/536] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:15.820786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:15.835565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:15.962642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebb15e-43bd-49b2-bcd9-67cdbe750338 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:15.998497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:15.999300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:15.999588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:15.999887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.011185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:16.012704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:16.030204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:16.031295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:16.032892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:16.032892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:16.032892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:16.032892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:16.032892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 263/537] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:16.039561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:16.082288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:16.156683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12922cb7-82cd-47b7-94ed-5991c79859d4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:16.209713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:16.210739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.211232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.211717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.212785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:16.213814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:16.215433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:16.230802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 275/538] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:16.236548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:16.364301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:16.422830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8cfa8-1cc8-4500-9142-1f57fdcf9e13 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:16.457052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:16.457847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.458133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.458414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.466184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:16.467631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-188a6706-2dbd-451e-84a0-b566fce02dbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:16.482087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:16.482913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:16.483730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 264/539] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:16.490198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:16.539141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd002cc-7c01-4c13-96d6-547daf5b22b6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.539452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd002cc-7c01-4c13-96d6-547daf5b22b6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.539710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd002cc-7c01-4c13-96d6-547daf5b22b6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.541725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 265/540] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:16.548914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.549203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.549488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.584834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:16.586049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:16.617538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:16.636884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6259d6b-9212-4a0f-a411-b026c28f9119 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:16.685390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:16.686643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.692215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.692513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.693340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:16.693667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:16.695142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:16.712131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 266/541] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:16.717377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 276/542] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:16.722994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:16.749692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:16.819657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:16.911585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f338e5-24ce-425c-b4b3-bbed72262ccf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:16.925407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bddcb6-6847-4632-a357-6ae77aab23a9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:16.958270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:16.959265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:16.959736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:16.960218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:16.961260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=05e67d8adf624d458533c80507374d78) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:16.962591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:16.964555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:16.978118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 267/543] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:16 2022] GET /identity/v3/role_assignments?user.id=05e67d8adf624d458533c80507374d78&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:17.002553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:17.003724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.004226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.004709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.025013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:17.033942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:17.034894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:17.035658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:17.038207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 277/544] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:17.044170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:17.186833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f8fa3c-eaa2-4784-82f9-b94369b9baf5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:17.202363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:17.225805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:17.226491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.226781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.227060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.227912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:17.228278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:17.230530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:17.241313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 268/545] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:17.247387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:17.262288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:17.392722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86fda2-f6f2-4040-8cf1-a97bb8990efb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:17.444738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:17.445784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.446229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.446651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.461465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:17.463533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e50b45ee-072c-489b-bd55-7451e68a7cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:17.482194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:17.483260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:17.484363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:17.484363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:17.484363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:17.484363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:17.486214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 278/546] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:17.491764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:17.505447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ad37-47f0-4bc6-b6c8-13fc4fe55df7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.506063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ad37-47f0-4bc6-b6c8-13fc4fe55df7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.506490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ad37-47f0-4bc6-b6c8-13fc4fe55df7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.508539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 279/547] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:17.518250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.518542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.518825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.543677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:17.544967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:17.644202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:17.656312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c90353-0479-48a2-a124-adc290a4139d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:17.680301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 280/548] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:17.691590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:17.692366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.692660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.692944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.693677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:17.693989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:17.695413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:17.708654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 269/549] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:17.714248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:17.725649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:17.784213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:17.880968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d2ed5-2e33-4aac-a219-e8f6ef71268a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:17.917634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:17.918450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:17.918744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:17.919023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:17.927017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:17.928527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:17.941984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdee755-6805-4a0e-af24-1a45648fdb92 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:17.947897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:17.949273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:17.950323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 281/550] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:17.951383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.003665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.046688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.047573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.047887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.049432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.075759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:18.089843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:18.090732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:18.091699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 270/551] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:18.095837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff289ed-4604-4590-b09e-f8646e688022 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.097951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.167879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.168583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.168873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.169149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.170010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:18.170319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.171808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.206367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 282/552] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:18.211633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.296055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526669c7-92d9-4624-b848-1042dc47d056 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.299675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.325357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f866b-4d2d-4927-ac0c-04bc6550c447 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.333572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.344737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.345955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.346836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.347306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.348360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:18.348863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.351259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9a27d3-6c40-470b-b4f3-643be6514ef4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.363208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 271/553] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:18.364921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.365266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789fbef-8ef3-49b5-a34d-974fcfc14c7e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.365586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.365879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.366164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.366408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789fbef-8ef3-49b5-a34d-974fcfc14c7e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.366882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789fbef-8ef3-49b5-a34d-974fcfc14c7e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.372952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 272/554] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:18.372952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789fbef-8ef3-49b5-a34d-974fcfc14c7e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.446892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.448749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecb679-7a93-4d45-af1e-6510c7570df6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.506951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789fbef-8ef3-49b5-a34d-974fcfc14c7e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.543715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.544556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.544848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.545156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.556842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.557625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 283/555] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 13:42:18.558485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.558885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.559176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.559453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.572966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.576217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:18.577408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-23e97475-9aac-42bf-8498-8634c6626c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:18.584602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:18.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:18.586577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:18.586577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:18.586577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:18.586577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:18.586577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 273/556] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:18.587027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740041-d3a4-4c6b-b211-4dbcd2a61dce None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.587325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740041-d3a4-4c6b-b211-4dbcd2a61dce None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.588160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f740041-d3a4-4c6b-b211-4dbcd2a61dce None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.593653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 274/557] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:18.593653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740041-d3a4-4c6b-b211-4dbcd2a61dce None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.601910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.656386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 284/558] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:18.657654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.710328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f740041-d3a4-4c6b-b211-4dbcd2a61dce None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.748594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.749385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.749668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.749970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.764076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.765456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.778703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e554bd-26dd-4c55-9b45-602b917c9493 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c61fdf06-b9df-4239-99c5-29d5da85c539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:18.782712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:18.788901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 275/559] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.789798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.802757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:18.803951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:18.816268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.816934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.817219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.817494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.818222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:18.818525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:18.819925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:18.823603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.833288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 285/560] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:18.833985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.867947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:18.884040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 276/561] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:18.885231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:18.945866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4caeb-8dbe-49f4-9109-0bd591bdc3d0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:18.982009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:18.982802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:18.983083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:18.983357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:18.994083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:19.001774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.002984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:19.003874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 286/562] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.004565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.014899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab603569-a417-404b-a768-ae6cbdc42e9e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.050199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.050860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.051148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.051427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.052190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:19.052499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.053864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.066583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.069669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 277/563] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:19.070403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.118362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8b53c-4e72-44a8-a963-049f162a5403 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.125138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.154676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.155465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.155753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.156070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.164331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.166405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.175931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d33aa-53ab-4536-8e2a-c24b05d83141 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:19.181519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.182612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:19.183548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 287/564] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.184790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.218269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.219099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.219386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.219670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.231000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:19.239173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.240324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:19.241148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 278/565] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.241900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.299616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f13756-a9ec-41cb-8629-410e7b753ffb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.318111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.340159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.340914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.341115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.341392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.342288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:19.342463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.343838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.348273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86fef5-379a-4e0b-a9f7-1511e16a18e3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.352301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 288/566] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:19.352900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.364129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.390586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.391380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.391661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.391977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.398616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.400620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:19.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.416379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:19.417321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 279/567] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.417791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.460715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e983432-d471-4ad0-b2c3-a374f27a72b1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.496904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.497579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.497869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.498143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.499000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:19.499306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.500776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.516977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a68118-a265-474c-ac1f-32223e486441 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 289/568] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:19.521151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.550676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.551265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.551552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.551853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.552597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:19.552905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.554264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113edd0-e040-47c5-bea7-cd501d9bed23 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.555158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.563176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 280/569] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:19.564531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba63fcb-31c5-4419-875c-a52fde544151 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.564824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba63fcb-31c5-4419-875c-a52fde544151 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.565108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba63fcb-31c5-4419-875c-a52fde544151 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.566321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 281/570] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:19.567009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba63fcb-31c5-4419-875c-a52fde544151 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.599372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.628430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec825b54-a97a-4da2-9a28-05c3a12c1ea7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.662561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.663352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.663637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.663949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.672851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba63fcb-31c5-4419-875c-a52fde544151 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.674568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.676068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:19.693846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.694721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.695584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:19.695584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:19.695584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.695584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:19.695584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 290/571] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.695887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.707977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.708637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.708923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.709204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.709991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:19.710305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.711717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.730257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 282/572] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:19.730915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.785251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.798316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473b3c1-272f-4ad2-8b96-bc7ec0c9b065 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.822823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:19.832673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709627e3-9a9b-4e55-8fa7-7c1d87f555c2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:19.841996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.843201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.843702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.844223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.856332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.857969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.871064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:19.871942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.872233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.872518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7bc89994-d9c1-4eae-bd4f-8f8007c4574f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:19.876299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.877144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.878000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:19.878000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:19.878000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.878000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:19.882232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 291/573] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.883718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:19.883853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:19.884084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:19.884408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:19.885740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:19.901500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:19.902946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:19.904327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:19.905186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:19.906038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:19.906038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:19.906038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:19.906038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:19.910833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 283/574] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:19.911899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:19.979369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 292/575] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:19.980276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.019311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80974a24-4701-4dac-9a11-31ea6eee052c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.052836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.058360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.059037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.059444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.059594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.060375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.060691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.062177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.073783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 284/576] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.074634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.080232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.100606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e5d9d-c8e1-4450-9968-035b4be93a48 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.137045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.137702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.137991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.138270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.139102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=f0bf046435a84f7592fd022c03eda0a1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.139420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.140912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.146217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 293/577] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:19 2022] GET /identity/v3/role_assignments?user.id=f0bf046435a84f7592fd022c03eda0a1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.146992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.187978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de85d8-5463-4693-9824-70e1674ce118 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.229092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.229772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.230064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.230350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.231725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.232076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.234192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.253751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 285/578] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.253751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.264943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6a32-2268-4bbd-b5d3-84172ee4efbd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.303712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.304429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.304891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.305182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.306008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.306358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.308034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.309953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.323744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.327797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 294/579] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.328060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.366370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042689-ce13-4b12-bf0b-5615a744923f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.403169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.403999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.404290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.404573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.414818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:20.423381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:20.424562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 286/580] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:20.425406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.434372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abe486-9421-4a43-b1b4-6d5649b193d8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.468991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.469791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.470084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.470366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.478558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.480074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:20.495399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:20.496435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:20.497358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:20.499717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 295/581] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:20.500611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.542767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b6705-1740-430e-b5da-1938b9d9f0b0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.567623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.569241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.583130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.583887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.584182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.584477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.591801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.593287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:20.608514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:20.609412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:20.610315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 287/582] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:20.611461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.621836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd94095-db22-416d-908f-74ae162700ad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.658526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.659325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.659614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.659923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.668821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.670277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:20.686000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:20.686953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:20.687898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 296/583] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:20.688465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.722261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56018b1-486c-4c8f-9dc5-512e0c2f7172 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.760163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.760833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.761123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.761398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.762147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.762455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.763810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.776836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 288/584] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.776836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.801762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86c07-293f-4b75-80a4-4b7975d2ac91 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.817195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.821458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:20.841998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.842605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.842889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.843164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.844037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.844346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.845768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.868328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 297/585] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.868328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.882178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a748ea9-75f6-4ad7-af5a-cae8c42ae220 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:20.919356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:20.920054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:20.920347 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:20.920630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:20.921431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:20.921748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:20.923314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:20.942202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 289/586] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:20.942202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:20.978256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c0b9e-a301-4741-ba0b-d3050332f31a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.014795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.015476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.015764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.016093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.016938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:21.017262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.018697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.035675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 298/587] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:21.036511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.054512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72319db9-aae5-44b9-b6ff-0b4e46970fd6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.063591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.067173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.093491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.094329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.094690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.095066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.105272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.106717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:21.124646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:21.125535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:21.126394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:21.126394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:21.126394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.126394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:21.126394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 290/588] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:21.126675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.145251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371aa43e-f43a-453a-8930-a90ec189e558 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.184574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.185357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.185645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.185924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.196253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.197848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:21.217311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:21.218247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:21.219087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:21.219087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:21.219087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.219087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:21.219087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 299/589] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:21.219444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.243233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6b91e-f465-46b3-838f-386e27cb2037 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.280391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.281282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.281565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.281847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.297921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=5bb056b4578442038d33d4fdf9de9584, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.300544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.312682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.316156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 291/590] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:21 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:21.317144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.330209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.348273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f410f-4cdd-477b-a65f-f66f5a717a5b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.401538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.402916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.403958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.404276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.420623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.422127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:21.450812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:21.452234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:21.453102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:21.453102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:21.453102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:21.453102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:21.453352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b9577c-338d-4bc0-a420-c7737340eab7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.456458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 300/591] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:21.457521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.499952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.500848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.501246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.501680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.502696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:21.503301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.508273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.525859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 292/592] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:21.527312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.593988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc2a43-b3e1-4822-b149-0f63bfd9a771 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.608600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.634148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.634940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.635304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.635733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.636669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:21.637059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.638601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.640320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.646099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a009c4-a649-4884-b3dd-5feb975a5d81 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.652551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 301/593] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:21.657862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.683118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.683911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.684275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.684630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.685438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:21.685840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.687279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.699827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 293/594] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:21.700667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.774529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa907e2-5f08-43cb-8d22-8d0bab5b4cde None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.804277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4344f8-ba1a-4c62-905c-897ea013e88f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.813127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.814022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.814319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.814601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.824125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.825597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.833722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.842251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:21.842873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:21.844762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:21.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:21.848358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:21.848358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:21.848635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 302/595] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:21.848798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:21.848942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.855846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:21.855846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:21.869345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:21.871375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 294/596] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:21.873646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:21.911483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:21.962906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29757bc-bf50-4945-ba3c-b7c98c5febcb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:21.977067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9479fac-9cd6-4baa-875d-e3852d6910fa None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.001487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.002742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.003200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.003647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.012399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.013528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.014224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.014457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.014583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.014783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.015692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:22.016041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.017467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:22.031654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:22.032777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:22.033873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:22.040345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 295/597] 10.222.0.24 () {56 vars in 1139 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:22.040345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.040954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 303/598] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:22.042203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.086632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.142574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6609919-b9ea-4341-9124-b45756a5ef5c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.156199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62f0a8-d29c-4178-a6fd-08f044e11f1c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.173703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.179058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.179787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.180747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.180747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.181772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:22.182422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.183512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.199849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 296/599] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:22.199849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.205022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.206097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.206593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.207079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.208295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:22.208819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.210598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.231587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 304/600] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:22.233328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.310012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf174-b7ed-4774-9f6c-6340c4b3a917 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.317113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.345274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.346103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.346394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.346676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.349931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c063412e-36a5-4d0b-8fdf-37af2c6d1777 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.358378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.359952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:22.377573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:22.378487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:22.379391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:22.379391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:22.379391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:22.379391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:22.379391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 297/601] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:22.379676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.389457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.390693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.391208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.391693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/9da57903126d421b8ac6c0e26b3fb3b2/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.412379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=9da57903126d421b8ac6c0e26b3fb3b2, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.414656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.439845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 305/602] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:22 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/9da57903126d421b8ac6c0e26b3fb3b2/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:22.440947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.470868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.532302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d8af1-b594-4d34-93ae-693fe7dc0465 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.576304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36070-fc54-416a-832e-4d1e6588a7ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.589890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.591985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.592548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.593031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/eb86cbb65b734fc69bf7604414f8c9cf/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.617966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=eb86cbb65b734fc69bf7604414f8c9cf, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.619673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d9946e-c28f-4ba9-8068-3e53f09d7daf None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.632264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.633212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.633505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.633783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.635061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:22.635372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.637376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.643237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 298/603] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:22 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/eb86cbb65b734fc69bf7604414f8c9cf/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:22.645058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f641e-495c-4383-8aca-2177eab8533f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.645577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f641e-495c-4383-8aca-2177eab8533f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.646823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f641e-495c-4383-8aca-2177eab8533f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.651449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 299/604] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:22.654178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 306/605] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:22.657208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.657521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.657781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.659348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.683396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:22.684587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:22.700210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.780672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 300/606] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:22.803272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.817309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:22.865949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d2f9f-43fb-4f60-9d75-104a4e8e6194 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.925081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9404fb-ac02-410e-98c7-2f0cb6bc10af None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:22.952639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.953387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.953685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.953968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.965129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:22.966679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:22.969817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:22.970858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:22.971336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:22.971797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:22.983542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:22.984496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:22.986245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 307/607] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:22.990023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:22.991245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:22.997101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:23.004784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:23.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:23.006848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 301/608] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:23.015022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.169062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cbd65-1fcd-4f72-b4d5-394f94567add None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.189018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7734faa-b896-424c-8874-fdf1967cc207 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.209531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.210132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.210420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.210702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.211459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:23.211770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.213696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.232173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 302/609] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:23.238141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.262779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.263841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.264329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.264801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.265821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:23.266330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.268731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.296092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 308/610] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:23.301720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.309182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:23.348202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:23.418882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3026cb-7355-463e-af48-0d54340dd0ef None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.458930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.459787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.460048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.460333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.473956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.483837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.501598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dda810-3f79-40b3-8093-ba1caf0727fa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:23.509929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:23.510870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:23.511732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:23.511732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:23.511732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:23.511732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:23.521008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 303/611] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:23.521888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.579039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.579982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.580268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.580544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/05e67d8adf624d458533c80507374d78/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.615361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=05e67d8adf624d458533c80507374d78, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.616870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.642067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 309/612] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:23 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/05e67d8adf624d458533c80507374d78/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:23.697333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d0ff6-0f8a-4a9c-bc85-f46a15f0a70e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.714006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:23.747274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:23.747986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.748279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.748564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.749311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:23.749625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:23.751006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1a8d4-07ac-4e51-bee8-084710a5b5e4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:23.773184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 304/613] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:23.780128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:23.809385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cc34-b26f-4e48-9097-1743d156360b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.809715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cc34-b26f-4e48-9097-1743d156360b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.809960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cc34-b26f-4e48-9097-1743d156360b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.811546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 305/614] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:23.817503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:23.818130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:23.818619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:23.848961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:23.850289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:23.876457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:23.964330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82313942-dced-4c3b-bf10-ecca4b06ccf1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:23.975610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 306/615] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:24.003272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.016877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.018456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.019041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.019551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.031472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.033594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:24.056725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:24.057853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:24.058766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:24.060274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 310/616] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:24.066681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.146412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886967e-c75e-47fc-839c-a82e00c6f4bf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.158822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:24.212877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.213703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.213965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.214248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.232819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:24.240692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:24.241716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:24.242597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:24.245236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 307/617] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:24.252816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.268922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429175a-9613-4f4b-89b4-f7501f618f40 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.304160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:24.346362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.347046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.347330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.350194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.351059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:24.351376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.352885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.389844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 311/618] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:24.412370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.468467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e4219-2468-4b94-b62d-37cc8a7d46c8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.535741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.536448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.536740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.537023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.537786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:24.538098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.539473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.550030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 308/619] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:24.553189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.599545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7b091-2ece-4268-84ad-e02edd0bf21c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.620099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:24.663610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.664446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.664733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.665008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.676156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.678043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:24.705592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:24.706604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:24.707512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:24.707512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:24.707512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:24.707512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:24.708246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 312/620] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:24.716099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.764267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:24.776559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1445f3-29e3-4000-9c36-ead0bd2771de None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.840534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.841343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.841630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.841911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.858874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.860367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.890296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097a9b0-2ca9-4440-8a88-3f60c983630b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:24.894543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:24.895401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:24.896361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:24.896361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:24.896361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:24.896361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:24.900888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 309/621] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:24.907639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:24.957623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:24.958333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:24.958623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:24.958903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:24.959655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:24.960007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:24.961465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:24.989153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 313/622] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:24.994460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.066667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.116872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3d74a-a2fa-41d5-b4a5-103e20f360d3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.138774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.161672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceee2937-0e41-425f-8066-4dcc01c7b8bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.179645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.180349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.180646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.181049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.181800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:25.182120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.183518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec3362-9610-4dc9-95df-abbe2605586f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.197239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 310/623] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:25.199088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b4c3-5841-413a-95b3-56b9f65e8ee4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.199381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b4c3-5841-413a-95b3-56b9f65e8ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.199657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e9b4c3-5841-413a-95b3-56b9f65e8ee4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.205088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 311/624] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:25.205862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9b4c3-5841-413a-95b3-56b9f65e8ee4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.215753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.216588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.216876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.217193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.224720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.226484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b213d562-c4d6-415f-8391-c2798a6b01af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:25.243921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:25.245889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:25.247927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 314/625] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:25.255881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.255881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.255881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.270160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.271591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.323120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e9b4c3-5841-413a-95b3-56b9f65e8ee4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.342296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 315/626] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:25.344213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.365763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.368151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.368893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.369597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.374751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.384832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.387565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9d706cce-e448-4a1d-934e-b97f9fd83628 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:25.408228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:25.409694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:25.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 312/627] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:25.413743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fe9-259f-49ab-b549-506244d5b84c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.413743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fe9-259f-49ab-b549-506244d5b84c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.413743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e2fe9-259f-49ab-b549-506244d5b84c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.416495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 313/628] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:25.417918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e2fe9-259f-49ab-b549-506244d5b84c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.435461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.463302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc55d6-4955-49f9-8f30-e9c53c6513cc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.503657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.503657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.503657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.503657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.504165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:25.504165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.505148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.522506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 316/629] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:25.522506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.545733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e2fe9-259f-49ab-b549-506244d5b84c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.582181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.582895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.583179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.583462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.595362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-26569e91-94f9-4dee-9e64-2380fc8a33bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:25.603987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:25.604928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:25.605740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 314/630] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:25.606942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.607040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.607323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.625491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:25.626595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01beaf15-ab80-4a13-ad32-6a9f701715cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.627053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:25.660554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.661232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.661524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.661806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.662657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:25.662976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.664449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.666801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.683307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.686219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 317/631] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:25.686999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.706938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 315/632] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:25.713541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.816068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571aa3d-f697-4fb1-b698-79b4d209746f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.843455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47c150-3f25-4cc2-b584-8de846d757d2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:25.857333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.858450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.858734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.859400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/f0bf046435a84f7592fd022c03eda0a1/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.877622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=f0bf046435a84f7592fd022c03eda0a1, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.879080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.881655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:25.882335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:25.882864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:25.883149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:25.884040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:25.884490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:25.886086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:25.894107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 318/633] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:25 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/f0bf046435a84f7592fd022c03eda0a1/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:25.894743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.910374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 316/634] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:25.910374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:25.968933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:25.988741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.061586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf4805-495b-4cb9-b957-615e10760685 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.081509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53a37a-8a1e-4ab4-8d2f-0142ffe576ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.106397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.107561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.108002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.108456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.142159 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.143580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.144058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.144494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.160025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:26.168941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:26.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:26.171149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:26.173304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 317/635] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:26.174252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.196995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.198819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.248351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.312521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a0dc5-ba91-425e-a848-721606846799 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.322832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 319/636] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:26.322832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.326420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.358621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.359440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.359811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.360184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.375513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.377178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-10dc04bf-16ec-471d-9625-738d626be227 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:26.399024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:26.401212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:26.402389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:26.402389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:26.402389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.402389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:26.402647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 318/637] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:26.405608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a327-3c02-40ab-9e87-8e9d4fc36ae2 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.405894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a327-3c02-40ab-9e87-8e9d4fc36ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.406172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9a327-3c02-40ab-9e87-8e9d4fc36ae2 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.408409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 319/638] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:26.409177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9a327-3c02-40ab-9e87-8e9d4fc36ae2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.449083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7ed04-ed48-41f1-8980-ee9c2c55332d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.492031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.495454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.495454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.495454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.495454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:26.495454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.496926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.509774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 320/639] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:26.509976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.521512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.537364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9a327-3c02-40ab-9e87-8e9d4fc36ae2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.581910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.583610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.583610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.583610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.597203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.599263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-840fee32-c38c-48ef-bbf9-5718ca0939bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:26.624388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:26.625295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:26.626183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:26.626183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:26.626183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.626183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:26.626183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 320/640] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:26.626901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.627189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.627476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.643953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bad715-7e71-4ca5-82d9-012e0c541cab None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.647746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:26.650320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:26.663303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.686737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.687410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.687703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.688030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.688784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:26.689086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.690470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.703845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 321/641] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:26.704691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.742763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 321/642] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:26.744192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.783631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:26.831482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5c90f-a186-47ad-b094-ed98c13e5ab0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.856047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7581db8-e71d-4459-91b1-cef782a9cdda None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:26.867469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.868935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.869222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.869499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.880193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.881643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.893007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:26.893722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:26.894011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:26.894287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:26.895020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:26.895326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:26.896691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:26.901241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:26.902197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:26.903085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:26.903085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:26.903085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:26.903085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:26.907110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 322/643] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:26.907674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.914075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 322/644] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:26.914075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:26.916351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.017611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d7910f-6399-4cdb-873b-27346e2f5c9d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.017857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e2ce-7413-411d-b713-1a55e0f5f9bc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.036282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.053341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.054146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.054440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.054726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.058812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.059614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.060073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.060264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.065884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:27.067456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.068968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:27.073901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:27.074826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 323/645] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:27.080990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:27.091657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:27.092638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:27.093545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 323/646] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:27.094891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.150675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.191177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a531bee-0e46-4bbb-b8e3-deacd2076ad4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.208289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82832dd-e019-4cee-9e94-0721d9b81f7d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.228862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.229948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.230455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.230948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.231935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:27.232352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.233776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.244775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.245564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.245848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.246124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.251637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 324/647] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:27.251637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.258257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.260290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:27.276071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:27.276954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:27.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 324/648] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:27.278277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.314742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.365193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466744-cc08-4981-bfb2-37aee4a037a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.382465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0959ea91-36bd-45ad-acae-1523c4d2a4ec None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.391093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.405139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.405748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.406156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.406439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.407323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:27.407639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.409185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.421898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 325/649] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:27.422760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.425159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.425840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.426123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.426399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.427272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:27.427582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.429067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.447346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 325/650] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:27.448168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.542652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9faa0c-2f6f-4d6d-a48f-e42cc980f405 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.558661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.567806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbf32a-2dd1-4898-8cd1-ad36ec55bb27 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.577691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.578501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.578798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.579070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.586453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.588995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:27.604141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:27.605082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 13:42:27.606021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:27.610223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 326/651] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:27.611078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.611455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.612092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.612376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.612652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.613444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:27.613942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.616317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.630683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 326/652] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:27.636890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.660013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.719538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546eb18-a130-402c-bfc5-994b9746a21a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.744364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276ed09-babb-41d9-9eba-c9595e85a495 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.756514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.757534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.758072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.758351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.770333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.771789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.781701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.782352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.782638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.782917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:27.790097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:27.791515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:27.792589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:27.792589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:27.792589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:27.792589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:27.792589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 327/653] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:27.794485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.810222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.861022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.862634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.900949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:27.906938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184f584-ebcb-48ee-9445-3461a2ca6988 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:27.944493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:27.945165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:27.945457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:27.945758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:27.947132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:27.947449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:27.949099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:27.967178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 328/654] 10.222.0.24 () {56 vars in 1137 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:27.968031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:27.982835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 327/655] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:27.982835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.040258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.082703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1171ec12-b514-4045-9817-7aee6793f207 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.094021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11528550-7ce8-4622-b619-9f64d233f384 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.119235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.120028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.120318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.120716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.121558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:28.121872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.123418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.129323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.140017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 329/656] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:28.140238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.140238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.140238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.140508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.140592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:28.140592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.140592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.155072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 328/657] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:28.156085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.164466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.273016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed97e07-ab73-4356-bf01-ed1b8e2bf745 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.274197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d7e4e-a911-4414-98d2-d5be8532ea37 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.289670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.313379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.314191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.314468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.314742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.315112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.315772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.316093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.316374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.326770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.328390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:28.347367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:28.348394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:28.349371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:28.349371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:28.349371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:28.349371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:28.353342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 329/658] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:28.354227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.398828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.400920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4307f2-2c9a-418c-9970-dd1af61d5203 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.424718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.472436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813cdc-9716-4f8e-8979-52ad54cd871d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.516959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.522006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 330/659] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:28.522140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.522140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.522140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.522351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02b190-f01a-420b-9a4d-700c12a2b77a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.522351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02b190-f01a-420b-9a4d-700c12a2b77a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.522351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02b190-f01a-420b-9a4d-700c12a2b77a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.522351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 331/660] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:28.522351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02b190-f01a-420b-9a4d-700c12a2b77a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.535466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.537089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:28.552384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:28.553524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 13:42:28.554695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:28.558453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 330/661] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:28.559282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.588388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.638614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02b190-f01a-420b-9a4d-700c12a2b77a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.656902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.674650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f70c9f-2099-43cc-99a4-383a66ccfba3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.675655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.676423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.676708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.676982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.677718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:28.678029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.679392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d905f07-78d7-4d36-b9ab-63b5cddeea69 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.692622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 332/662] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:28.693849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.694140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.694401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.711765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:28.713571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:28.716716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.717747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.718035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.718321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.730001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.731587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:28.751744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:28.753328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:28.754295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:28.754295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:28.754295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:28.754295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:28.754295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 331/663] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:28.754573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.792205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 333/664] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:28.792205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.857447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.880801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ebe4-7ed6-4cae-9118-30dabdfdf6ed None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.904990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470ed92-9e70-4c5b-9f08-5a7b9b958699 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:28.916489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:28.924287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.924958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.925241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.925761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.926359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:28.926669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.928108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.943469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:28.944300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:28.944590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:28.944871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:28.950821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 332/665] 10.222.0.24 () {56 vars in 1139 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:28.950821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:28.956635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:28.958103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:28.972112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:28.972975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:28.973854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 334/666] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:28.974341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.060788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b758fb3-0971-4c45-9bcf-ed322c796f28 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.088191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e3b84-270d-4e18-b45f-59b006e42b0e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.095571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.096260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.096548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.096823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.097544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:29.097851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.099214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.102426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.112539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 333/667] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:29.116690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.125273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.126487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.126780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.127058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.139453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:29.147197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:29.148125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:29.148963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 335/668] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:29.149746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.180400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.224051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b35e-a545-4415-9957-5789ec8c2c1d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.259595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.260590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.261070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.261357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.263461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cdd9d7-7c69-4b29-bb91-20f2f62cd1bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.299370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.300066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.300355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.300631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.301416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:29.301906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.303321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.317361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 336/669] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:29.317918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.331418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.340111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.341646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.412611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.426025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddca0c3-40df-4b69-9e75-e8a5e463b887 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.448620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 334/670] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:29.449293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.466674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.467472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.467756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.468072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.474761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.476228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:29.490285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:29.491176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:29.492101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 337/671] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:29.493146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.562631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.568888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2711e0-6ed6-4427-b623-5b2e0f5c173c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.598344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d58b7-67ab-4386-b37d-a4d8ebc46012 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.605184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.605860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.606152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.606854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.607619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:29.607967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.609723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.624612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 335/672] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:29.624612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.636016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.636686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.636974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.637252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.638469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=9da57903126d421b8ac6c0e26b3fb3b2) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:29.638791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.640585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.646823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 338/673] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/role_assignments?user.id=9da57903126d421b8ac6c0e26b3fb3b2&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:29.648691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.671903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.744822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41af5a-c4a4-4a44-9254-aed94b17907a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.783159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.784179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.784568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.784964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.797459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.799359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.806995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d6f9-9d7e-4f1f-95bc-70cfa7ceec72 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:29.827793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:29.828855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:29.829941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:29.829941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:29.829941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:29.829941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:29.831877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 336/674] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:29.832945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.848297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:29.860126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:29.860923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:29.861308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:29.861679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:29.862572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:29.862982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:29.866461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:29.877888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 339/675] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:29.879696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:29.963946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db317ea-6408-4ee2-8ce0-dcb6be05e91d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:29.991359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.001613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.002907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.003409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.003780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.011778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dbb8a6-23c6-4970-aaa4-364c6a3c72ac None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.015922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.017386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:30.035171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:30.036129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:30.036992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:30.036992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:30.036992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:30.036992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:30.041169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 337/676] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:30.041917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.052199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.053201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.053685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.054184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.055102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:30.056279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.058046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.070033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 340/677] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:30.075469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.109447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.151975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d83038-b211-4c71-aa4f-71ee74fbf4b6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.186438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.187123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.187518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.187663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.188517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=eb86cbb65b734fc69bf7604414f8c9cf) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:30.188947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.190396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.194235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b90c6-b8ed-4ace-b618-6712d1cee016 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.201983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 338/678] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/role_assignments?user.id=eb86cbb65b734fc69bf7604414f8c9cf&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:30.203306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.229733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.240921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.241984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.242452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.244010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.244989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:30.248561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.250240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.274363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 341/679] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:30.280699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.481715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc28e830-fe95-40dd-92cb-f26cffab08dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.534863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.535676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.536549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.536549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.537694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc7d8a8-f95b-4f3e-bf59-81a84a143449 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.545065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.546269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-938470f4-4040-4177-9565-7c35a7d51f85 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:30.562302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:30.563443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:30.564668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:30.566422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 342/680] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:30.567587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d4424-177a-474d-b9e3-7e22f416d81c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.568676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d4424-177a-474d-b9e3-7e22f416d81c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.568969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d4424-177a-474d-b9e3-7e22f416d81c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.570588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 343/681] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:30.572472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d4424-177a-474d-b9e3-7e22f416d81c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.583590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.599066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.600251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.600537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.600816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.609803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.611246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f107fc44-50c8-4d5c-a812-99e37809f824 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:30.631258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:30.632234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:30.633466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:30.636696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 339/682] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:30.638306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.638766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.639194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.653461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.659623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:30.661040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:30.726858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d4424-177a-474d-b9e3-7e22f416d81c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.764573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 340/683] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:30.765858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.789122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.790598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.792627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.793068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.793928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:30.794242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.795670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.814581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 344/684] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:30.815379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.894066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdce80-941f-4d56-8cd4-76ebb9287743 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.917202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.926937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc7208-57ab-4639-b8bf-513f6ce650de None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:30.932586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.933317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.933606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.933923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.934733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:30.935048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:30.936523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:30.944549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:30.955943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 341/685] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:30.956858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:30.978005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:30.978900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:30.979195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:30.979478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:30.990933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:30.998907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:30.999854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:31.000709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:31.003970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 345/686] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:31.004897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.084269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba882519-abeb-4095-aeb7-223cada7cb11 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.120326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.121379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.121876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.122361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.126834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7308e-c789-48f3-8143-a5338fc75422 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.164554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.165480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.165848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.166238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.167086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=05e67d8adf624d458533c80507374d78) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:31.167406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.168982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.172745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.176345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 346/687] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/role_assignments?user.id=05e67d8adf624d458533c80507374d78&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:31.177392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.200167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.202095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.220864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.295603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe0468-0232-4253-ae93-91e8c417b254 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.335036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.336164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.336689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.337725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.338835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:31.339365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.340123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 342/688] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:31.345324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.347622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.370724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 347/689] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:31.376873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.477003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.514699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58fbb1-16ce-42ac-820a-fb5a70988bcc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.558361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.559631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.560213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.560693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.566225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.573781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21df2-eba2-49d7-b4f1-565128608a85 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.575188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.577585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:31.598254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:31.599144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:31.600460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:31.600460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:31.600460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:31.600460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:31.602697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 343/690] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:31.608465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.612138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.614711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.614711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.614711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.628130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.629948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:31.655774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:31.656738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:31.657604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:31.657604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:31.657604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:31.657604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:31.657604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 348/691] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:31.663116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.740526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d7cdc5-25a4-4a86-b9de-002ca1dc33d3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.772069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.796676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cb9fd6-340c-4f2e-9b7d-2efcc100a90b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:31.804845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.806290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.806985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.807560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.808532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:31.809054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.810847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.832685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:31.833355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:31.833643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:31.833926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:31.835227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:31.835546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:31.837203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:31.840242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 344/692] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:31.845850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.849794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 349/693] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:31.856979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:31.880134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:31.998580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9864cc-7c0e-4d98-8292-f819d31bf3e7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.004695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d09ef-7fbf-4fd8-84e6-84970af5da84 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.049654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.049654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.049654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.049654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.053732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.055186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:32.072332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:32.073307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:32.074236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 350/694] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:32.079703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.095206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.096120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.096414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.096750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.108598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.110209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:32.140033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:32.141154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:32.142229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:32.144381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 345/695] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:32.150933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.201751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:32.213483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:32.257204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce7504-49fc-4d84-abb4-b05549febb11 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.268787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114312b0-18e4-4c08-b684-de52032a52da None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.294099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477861bf-628d-484b-904c-edc93529d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.294897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477861bf-628d-484b-904c-edc93529d64a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.295254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477861bf-628d-484b-904c-edc93529d64a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.295645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477861bf-628d-484b-904c-edc93529d64a None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.296481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477861bf-628d-484b-904c-edc93529d64a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:32.296797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477861bf-628d-484b-904c-edc93529d64a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.298335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477861bf-628d-484b-904c-edc93529d64a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.311883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 346/696] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:32.312190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.312878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.313169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.313301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52b34-f1c0-4fd3-aea2-454b02349b9d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.313462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.313555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52b34-f1c0-4fd3-aea2-454b02349b9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.313724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e52b34-f1c0-4fd3-aea2-454b02349b9d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.314296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:32.314611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.316125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038f29d-9606-49ae-b0e7-a12644c4a441 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.320166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 347/697] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:32.321286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e52b34-f1c0-4fd3-aea2-454b02349b9d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.339917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 351/698] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:32.341263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.341556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.341839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.366494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:32.369111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:32.447720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 352/699] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:32.448957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.458384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e52b34-f1c0-4fd3-aea2-454b02349b9d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.470766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:32.515599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.516521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.516965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.517896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.519179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=f0bf046435a84f7592fd022c03eda0a1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:32.519650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.521557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:32.522803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.533093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 348/700] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/role_assignments?user.id=f0bf046435a84f7592fd022c03eda0a1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:32.534224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.610617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf0468-3edd-482a-b41d-d4b892e0d5fa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.666154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e6951-27c4-443e-995e-be180d32f23a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.680990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.681894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.682200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.682590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.712932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.714235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.714815 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.715294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.716444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.718536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:32.740942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:32.743795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:32.745705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 349/701] 10.222.0.24 () {56 vars in 1129 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:32.750289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.799393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:32.870884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.872393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.888653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3eab0-3d6c-45cd-bea7-3105be66e15c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:32.927181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:32.927780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:32.928102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:32.928387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:32.929107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:32.929418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:32.930828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:32.935982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 350/702] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:32.945731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:32.954026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.003504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 353/703] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:33.010466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.081907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04113cc4-f16b-45e8-a068-65c588cf9091 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.084172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.117562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.119015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.119300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.120068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.120795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:33.121107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.123114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b67928-3111-4317-92a3-bd6c6a3ebd87 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.129996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c48f7-38a0-42d6-be7f-b4f945e8bcc6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.136919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 351/704] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:33.138972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb1d9-37bf-4473-ab78-33a41e48f6b2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.139279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb1d9-37bf-4473-ab78-33a41e48f6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.139574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cb1d9-37bf-4473-ab78-33a41e48f6b2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.145214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 352/705] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:33.147073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.147368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.147657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.166332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:33.167790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:33.170679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.171495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.171855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.172149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.184461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.185847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:33.204801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:33.205737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:33.206613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:33.206613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:33.206613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:33.206613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:33.206613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 354/706] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:33.221286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.242049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.252632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 353/707] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:33.276640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.388505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22325984-2bd0-4821-97f0-8fc963456f26 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.390070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.427376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.428222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.428507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.428783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.440846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.442227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.460243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a62c1c-8f78-40bf-8646-73ffee93c746 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:33.461703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:33.462984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:33.464344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:33.464344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:33.464344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:33.464344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:33.467796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 354/708] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:33.469065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.502785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.503466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.503935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.504221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.505298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:33.505622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.507265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.522026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 355/709] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:33.532866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.586586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f13e-7381-4cc2-b9bb-b6a671a56f1c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.622888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.623565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.623870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.624151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.624881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:33.625188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.626554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.638937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.643490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 355/710] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:33.644972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.656325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:33.702426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340a46a-8d63-4d68-b13b-350277672cb3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.759320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eef681f-469e-4592-af38-fb2563520c5a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:33.773375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.775169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.775920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.776404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.783535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.785271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:33.814726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:33.815682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:33.816638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:33.823363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:33.824347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:33.824836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:33.825362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:33.826294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:33.827973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:33.833159 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 356/711] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:33.834268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:33.862670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 356/712] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:42:33 2022] POST /identity/v3/users => generated 254 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:33.998330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28571fcb-476e-4ca4-b7ad-613b4fa19d2d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:34.032431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:34.042783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:34.043910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.044397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.044919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.045943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:34.046463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:34.048174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edb0b3-27b0-47b6-90bb-ee58dcff48a2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:34.061657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 357/713] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:34.070365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:34.161794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:34.186521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe597-de58-4b69-9980-3cb9079dcf8d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.186922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe597-de58-4b69-9980-3cb9079dcf8d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.187100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe597-de58-4b69-9980-3cb9079dcf8d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.188755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 358/714] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:34.196102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.196411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.196668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.212784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1ae3b-f0db-4e27-9ae5-fb5b166d7893 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:34.217004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:34.218198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:34.253095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:34.254220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.254710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.255234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.256307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:34.256904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:34.259299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:34.271946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 357/715] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:34 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:34.319509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 359/716] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:34.348311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:34.490205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4deecd-de7c-40d7-bb34-0e18742f6b83 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:34.529574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:34.539418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:34.540490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.540958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.541392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.542268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:34.543925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:34.557041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:34.558270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:34.559311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:34.561855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 358/717] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:34 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:34.567367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:34.702769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98aa0153-52df-4d58-b18f-b7755dd3a025 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:34.730779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:34.747971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:34.748645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.748931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.749209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.749919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:34.750226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:34.751600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59bf9c-7f0a-4f1a-9744-455707699063 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:34.756374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 360/718] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:34.762909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:34.877080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405a1a8-bf10-45b4-9fcb-e02471c7fb57 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:34.919741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:34.920442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:34.920728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:34.921009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:34.921793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:34.922105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:34.923547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98182ab2-7980-4885-a6f6-d0cf356b5df3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:34.928794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 359/719] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:35.561002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcacd4-2095-4254-8e25-ec5e2b1d5c6d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:35.561350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcacd4-2095-4254-8e25-ec5e2b1d5c6d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:35.561575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcacd4-2095-4254-8e25-ec5e2b1d5c6d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:35.563209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 361/720] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:35.568187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:35.568648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:35.568817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:35.589011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:35.590207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:35.654488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:35.679456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 360/721] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:35.719335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcacd4-2095-4254-8e25-ec5e2b1d5c6d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:35.870984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcacd4-2095-4254-8e25-ec5e2b1d5c6d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:35.918917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:35.919722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:35.920042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:35.920324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:35.940678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:35.948665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:35.949585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:35.950847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 362/722] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:35.959403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:36.090191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec445-21c4-4583-988f-7d2f30167792 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:36.127619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:36.128249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.128541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.128826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.129590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:36.129904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:36.131289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a3471-8e18-428b-9b99-f1d4685b85c7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:36.141150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 361/723] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:36.149263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:36.215742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:36.283609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc2181-c00f-4995-96d9-a3824a429ee4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.283944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc2181-c00f-4995-96d9-a3824a429ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.284233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc2181-c00f-4995-96d9-a3824a429ee4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.285868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 362/724] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:36.292381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1b20e-7ef5-4604-842f-fb4e1ca07e06 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:36.292735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.293025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.293302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.321338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:36.322831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:36.358091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:36.358892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.359174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.359452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.378472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:36.379846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:36.413145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:36.414158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:36.415860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:36.415860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:36.415860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:36.415860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:36.416106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 363/725] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:36.417894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:36.422772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:36.481438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 363/726] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 13:42:36.541984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:36.612176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:36.622187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563ad6e-b10a-4077-8347-21dc1a6b81fe None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:36.693083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:36.693680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.693972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.694257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.695000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:36.695314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:36.697020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:36.715545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 364/727] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:36.720610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21de0d-74d9-453f-bfce-028b72084314 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:36.721975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:36.788847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:36.790419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.790867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.791292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.792205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:36.793745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61db3db-04bb-404f-9d4c-95f241d200b5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:36.821359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 364/728] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:42:36 2022] POST /identity/v3/services => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:36.842120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8cb53-b3df-40a0-8e6f-020cf0491af4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.842682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8cb53-b3df-40a0-8e6f-020cf0491af4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.843120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8cb53-b3df-40a0-8e6f-020cf0491af4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.844926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 365/729] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:36.851091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918aff02-73bd-4bbd-b6b4-bcfc2d7f6080 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.851698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918aff02-73bd-4bbd-b6b4-bcfc2d7f6080 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.852828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918aff02-73bd-4bbd-b6b4-bcfc2d7f6080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:36.880422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-918aff02-73bd-4bbd-b6b4-bcfc2d7f6080 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:36.883867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-918aff02-73bd-4bbd-b6b4-bcfc2d7f6080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:36.922765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81294fbc-8971-4309-bb9f-f3ac71c20d38 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:36.950409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:36.995509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:36.996989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:36.997491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:36.997979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.008170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 366/730] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:37.017108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.018796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:37.034693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:37.035927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:37.036881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:37.039456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 365/731] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:37.043074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95328b8-deae-478d-87c3-7718ec429637 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.043264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95328b8-deae-478d-87c3-7718ec429637 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.043662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95328b8-deae-478d-87c3-7718ec429637 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.044419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.047844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 367/732] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:37.049665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95328b8-deae-478d-87c3-7718ec429637 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.059986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.161088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2cf0-c8d3-4be6-84af-70b9a1b945dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.168740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95328b8-deae-478d-87c3-7718ec429637 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.199443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee0733-e272-46af-9293-08c42788441e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.200282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee0733-e272-46af-9293-08c42788441e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.200627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee0733-e272-46af-9293-08c42788441e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.200850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee0733-e272-46af-9293-08c42788441e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.212151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee0733-e272-46af-9293-08c42788441e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:37.215287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.216264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.216993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.217530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.218943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:37.219555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cdee0733-e272-46af-9293-08c42788441e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:37.220249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:37.221257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:37.222109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:37.224427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.226287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 366/733] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:37.228507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.228803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.229087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.247757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 368/734] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:37.249048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.251987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:37.253873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:37.282954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.326306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.339902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 367/735] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:37.342085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.385193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12287f19-c136-4fd1-8cf5-c75baf2fb7b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.445589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.446394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.446688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.446973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.448021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:37.448345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.450170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.466249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 369/736] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:37.466249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.467655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77b9644-16a5-49dd-8687-2d267b6fa22b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.516773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.517437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.517727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.518011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.577720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.590560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.593028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.594575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.597490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb293e-655c-411e-9aec-2eb917e1e959 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.637021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.638313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.638817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.639356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.651357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:37.659851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:37.660866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:37.661716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:37.669642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 370/737] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:37.670881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.697364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 368/738] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 13:42:37.698230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.784437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3745edd-2cc5-47a5-8b68-19b90e4080e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.802579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.807193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287053-706f-4edc-b2c6-f87315200681 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.822889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.823690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.824010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.824289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.835441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.837571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.844254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:37.848342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:37.849028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.849319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.849601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.850368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:37.850682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:37.852274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92495f-0833-49a7-8e55-7cda886ec2dc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:37.856097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:37.856974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:37.857831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:37.857831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:37.857831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:37.857831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:37.862099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 371/739] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:37.863003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.866809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 369/740] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:37.869261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c80ae-aaa8-43a1-b8bb-1adc71875061 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:37.869565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c80ae-aaa8-43a1-b8bb-1adc71875061 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:37.869854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c80ae-aaa8-43a1-b8bb-1adc71875061 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:37.875165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 370/741] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:37.875165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c80ae-aaa8-43a1-b8bb-1adc71875061 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:37.970888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc3d8f-e0c0-4e4e-948a-f6553f2ba1bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:37.991407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c80ae-aaa8-43a1-b8bb-1adc71875061 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.008248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.009898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.010179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.010457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.022566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.024066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.031415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.032704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.033072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.033386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.034201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:38.034567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.035973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319726c0-4753-46f9-8d5f-7d8d1897fc5b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:38.044523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:38.045487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:38.046396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:38.046396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:38.046396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:38.046396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:38.046396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 372/742] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:38.046680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.052568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 371/743] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:38.056412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.056989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.057993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.078865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:38.080366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:38.098725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.103042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.164004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 372/744] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 13:42:38.164004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.175600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60702aa8-cf01-4388-9c8a-e7defe2c5dcf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.213411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.214226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.214523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.214805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.226852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.228803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:38.248141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:38.249084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:38.249929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:38.249929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:38.249929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:38.249929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:38.249929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 373/745] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:38.250206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.299427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd76e1f-b976-4b22-af95-546af9db7628 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.336736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.337447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.337744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.338029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.338762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:38.339078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.340489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.358383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 373/746] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:38.358383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.372572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2c5e1-a2cd-45b6-945d-d46b7ea289d5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.374154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.384693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.411704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.412544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.412840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.413131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.421719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.423166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:38.438253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:38.439181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 374/747] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:38.440559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.476436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c641d-4c62-4b98-b178-8f8c6930c12e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.515641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.516338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.516626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.516906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.517634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.518995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.535250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 374/748] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:42:38 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:38.536375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.573316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48860732-b9df-45a3-bf64-5ba3748f02b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.624962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.635935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.636991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.637501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.638070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.639029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:38.639565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.641269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.660130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 375/749] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:38.661463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.667898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b644f4-e25c-42b9-8544-3c2bd81e3f7b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.688077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.715493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.716372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.716666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.716948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.727590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.729525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:38.748322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:38.750237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 375/750] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:38.753258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.817233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5087b-192d-46bb-b048-27fa5599624c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.853685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.854289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.854582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.854868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.855886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:38.856177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.857671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.881122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 376/751] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:38.882552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.890931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3b9c0-e323-4f71-9b1f-fa8eaad07878 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:38.931536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:38.938929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:38.940217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:38.940780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:38.941229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:38.951481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:38.953684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:38.974257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:38.975280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:38.976249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 376/752] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:38.976836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:38.984626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.016548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3cae3-41f6-4bfa-986b-a4a9a1e04cbd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.070054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.072163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.072455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.072742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.073563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:39.073884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.075341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.093770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 377/753] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:39.093770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.122052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf29c6-7675-41c8-b8bb-ea2477f03d2c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.172202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.173135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.173602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.174028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.204384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55d76d-158c-41d2-b79b-744cc1f9ade5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.244854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.245746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.246142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.246528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.247501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:39.247967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.249508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.253576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.256650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.272474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 378/754] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:39.273433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.275100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.277209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.397207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a567d0-f591-4a94-b4c2-54528c282640 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.404726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 377/755] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:42:39.405769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.447949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.449426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.449909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.450344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.451697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=f0bf046435a84f7592fd022c03eda0a1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:39.452181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.454491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.462450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 379/756] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/role_assignments?user.id=f0bf046435a84f7592fd022c03eda0a1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:39.463976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.518914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.530991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ae454-5fc0-47e5-ae2f-e238ebac59ad None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.546825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.570908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.572222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.572611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.573014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.584395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.586047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.604914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba004ef9-8347-4dce-bf20-e0efbbf20c00 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:39.607672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:39.608842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:39.609915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:39.609915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:39.609915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:39.609915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:39.611476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 378/757] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:39.618010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.661965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.664002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.664315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.664673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.689210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.693474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-643a49ed-385d-46f1-ac8e-88118f285b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:39.729548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:39.730517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:39.731385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:39.731385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:39.731385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:39.731385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:39.732295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 380/758] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:39.734526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ea0a-13ab-41bd-9f6c-da3e7a093ea2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.734980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ea0a-13ab-41bd-9f6c-da3e7a093ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.735596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ea0a-13ab-41bd-9f6c-da3e7a093ea2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.743977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 381/759] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:39.745568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ea0a-13ab-41bd-9f6c-da3e7a093ea2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.763045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc35e1-3be4-401a-bd1c-210612dcb50e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.802656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.803532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.803842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.804182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.804940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:39.805681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.807587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca5a9e9-68ce-4765-8b8e-450e365c3094 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.820553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 379/760] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:39.822847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.823293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.823631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.833438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.847863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:39.849818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:39.893799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ea0a-13ab-41bd-9f6c-da3e7a093ea2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:39.906329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:39.934763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 380/761] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:39.936200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:39.940837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:39.941666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:39.942057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:39.942444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:39.943302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:39.943726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:39.945257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:39.972226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 382/762] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:39.978327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.048779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a9867-21e6-462e-ad68-b8f95a597dc3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.091619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.092931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.093402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.093831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.096144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:40.102280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.104277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.114618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04738d93-7de6-4284-9949-5ebc4b3f35c8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:40.118806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:40.119918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:40.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:40.127158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 381/763] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:40.128441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.163309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.164678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.165145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.166167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.175586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.177765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:40.198684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:40.199771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:40.200914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:40.203742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 383/764] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:40.204752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.247843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a035-9d0d-4ede-8862-62d409e60d28 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.248351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:40.289255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.290402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.290727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.291109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.292008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.293562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:40.310015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:40.311175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:40.312116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:40.313130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 382/765] 10.222.0.24 () {56 vars in 1129 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:40.314029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.343396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8435-06f7-4115-9d9c-7d395ae6c84a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.381807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:40.391734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.393218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.393828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.394361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.395575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:40.396069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.398584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.415735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 384/766] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:40.416884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.437089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e369e-0cd4-48bb-88bf-156b61d6bdc6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.485202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.486402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.486598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.486998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.488403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:40.489031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.490684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.521169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 383/767] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:40.522390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.540433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:40.553283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e91ec-4942-4b1b-9acb-25f281c38c2f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.604862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.606080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.607383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.607841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.608720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:40.609206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.610752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.618260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 385/768] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:40.619633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.641226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7acc81-04de-46a8-9214-90cc3cb83a51 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.661501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:40.685015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.685987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.686386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.686831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.689379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=5bb056b4578442038d33d4fdf9de9584, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.690946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.708305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 384/769] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:40 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:40.713009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.786033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b6c3a-cb1c-482c-9f5e-fcf9823933ba None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.868249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.869787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.870514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.870979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.874039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=6d80ba86a9bf4219a509d11f4617930a) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:40.874627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.876568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.888245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 386/770] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/role_assignments?user.id=6d80ba86a9bf4219a509d11f4617930a&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:40.913256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41b912-baa5-4ec8-9624-b15451348cfc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:40.957222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:40.958689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:40.959163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:40.959624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:40.960563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:40.961058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:40.962815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0346e8b-8e0e-48e2-9c78-2e1e6e5c8611 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:40.971881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 385/771] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:40.977847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:40.998772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:41.097337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736870d2-ecd0-4c0a-8a40-b7171dacca90 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:41.117214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972262e4-8333-489e-ad57-510d55469b4e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.118185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972262e4-8333-489e-ad57-510d55469b4e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.118403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972262e4-8333-489e-ad57-510d55469b4e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.120079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 386/772] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:41.125478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.125768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.126049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.150635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:41.151884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:41.172934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:41.173618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.173910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.174191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.175963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] RBAC: Adding query filter params (interface=public, service_id=9b7cca28e2c84c70b9e274b95af2b736, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:41.175963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:41.176725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:41.192315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 387/773] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:42:40 2022] GET /identity/v3/endpoints?service_id=9b7cca28e2c84c70b9e274b95af2b736&interface=public®ion_id=RegionOne => generated 194 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:41.197480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:41.232219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:41.262750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 387/774] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:41.284977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:41.365628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d2e54-c338-4d67-8039-03c239a15df3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:41.404219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:41.413714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:41.414949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.415237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.415548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.416252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:41.417656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:41.436057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 388/775] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:42:41 2022] GET /identity/v3/services => generated 772 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:41.481926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69249-f8f4-4b59-8351-5484f3f47c1b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:41.535331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:41.536198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.536486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.536764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] PATH_INFO: `/v3/services/network` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.537443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:41.538858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:41.557649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:41.558709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:41.559795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 388/776] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:41 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:41.565187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:41.725408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782f138-0b48-4a51-a8a8-892bca3f8188 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:41.770800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:41.771483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.771772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.772121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.772847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:41.773158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:41.774596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d3c0c-390d-45ec-bacd-d73ec7b320e1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:41.784038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 389/777] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:41.789909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:41.851297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:41.940219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643bf28-026f-47d2-a001-916d4012ab56 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:41.989927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:41.990645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.990878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.991160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.991907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] RBAC: Adding query filter params (type=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:41.992221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:41.994194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85546c54-47b6-4dec-98c1-5896634e1861 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:41.995472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8399-7e8a-48f0-a4ac-4f9cad51b1ee None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:41.995975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8399-7e8a-48f0-a4ac-4f9cad51b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:41.996132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd8399-7e8a-48f0-a4ac-4f9cad51b1ee None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:41.997890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 390/778] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:42.001966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 389/779] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:42.007906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd8399-7e8a-48f0-a4ac-4f9cad51b1ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.008410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.008742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.009079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.022548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:42.038912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:42.040228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:42.119163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 390/780] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:42.143650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.196648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:42.222446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd8399-7e8a-48f0-a4ac-4f9cad51b1ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.278047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e3c37-f8cf-4676-a4dc-e5bd4a70915a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.292864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.293981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.294528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.295019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.296210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] RBAC: Adding query filter params (interface=public, service_id=a6fb84cf9f4b4c32b66d36ff877a9308, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:42.296874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:42.298513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:42.312841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 391/781] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/endpoints?service_id=a6fb84cf9f4b4c32b66d36ff877a9308&interface=public®ion_id=RegionOne => generated 194 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:42.317831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.319348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.320194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.321024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.321320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.332422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:42.340639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:42.341578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:42.342831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 391/782] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:42.350324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.453395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed8fc4-1a87-4d10-bcef-bfb5dea733cf None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.468176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:42.474580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:42.489450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.490129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.490420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.490702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.491461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:42.491773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:42.495187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:42.507575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 392/783] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:42.515023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.581024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f43f2e-6294-46e2-b2d5-b762aeed7a01 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.620535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f699d34-bc50-4f93-a6ae-42ab05d548dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.647924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.649143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.649441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.649721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.650427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:42.651808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:42.663941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 392/784] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/services => generated 772 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:42.689473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.690310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.690608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.690895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.708221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:42.709733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:42.732686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:42.733645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:42.736039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:42.736039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:42.736039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:42.736039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:42.736039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 393/785] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:42.739771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.868085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00188d04-3930-4fca-ae74-2daf403b99d1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:42.907084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:42.907759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:42.908076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:42.908358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:42.909095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:42.909408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:42.910784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:42.918821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:42.924706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 393/786] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:42.932926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:42.940421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:43.051133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26df06-ae3b-439f-9f14-751f5cb655b7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:43.093586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:43.094409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.094700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.094981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.113002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:43.114462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:43.132495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:43.134144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:43.135066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:43.137092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 394/787] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:43.141938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:43.309322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dff8bd-9244-4778-837c-abf4ad56677d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:43.360853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:43.361549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.361844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.362130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.362986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:43.363304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:43.364770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:43.387867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 394/788] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:43.393309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:43.519268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:43.582657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a046381-44b5-4671-9508-39eb41586fc0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:43.634192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:43.634996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.635288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.635573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.649438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:43.650835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b5110263-3332-454b-8ca2-1d5464adac43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:43.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:43.673362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:43.674249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:43.674249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:43.674249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:43.674249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:43.674249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 395/789] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:43.679188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:43.761404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996a9b-f781-4041-b911-4514778ca832 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.761731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996a9b-f781-4041-b911-4514778ca832 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.761986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78996a9b-f781-4041-b911-4514778ca832 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.763493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 396/790] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:43.777225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.777531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.777698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:43.777799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.810522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:43.811760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:43.837727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92926931-51f4-4386-9234-f93aff1af013 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:43.889438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:43.890120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:43.890405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:43.890684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:43.891423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:43.891733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:43.894086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:43.923704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 395/791] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:43.929158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:43.974702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 397/792] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:42:44.021777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.059228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e42c1e-cd99-40ab-9b4a-a4c632c2bb95 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.075890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.078119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.097842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.098639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.098929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.099208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.106460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:44.107972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-44c0a7b6-9326-4dc4-a1da-eaddc943bf47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:44.122110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:44.122977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:44.123866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 396/793] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:44.124702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b214a0-8f11-4266-9a2d-c99985ee2f1a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.124806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b214a0-8f11-4266-9a2d-c99985ee2f1a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.125093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b214a0-8f11-4266-9a2d-c99985ee2f1a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.130407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 397/794] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:44.131848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382f72-48c8-426a-bdc8-aeb2feb879c0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.132147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382f72-48c8-426a-bdc8-aeb2feb879c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.132687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12382f72-48c8-426a-bdc8-aeb2feb879c0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.134453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 398/795] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:44.135286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12382f72-48c8-426a-bdc8-aeb2feb879c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.143742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cef9ff-5abe-45a4-85a6-b5306636409c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.255700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12382f72-48c8-426a-bdc8-aeb2feb879c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.293565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.294233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.294529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.294808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.295599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:44.295950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:44.297394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc002ed6-3d75-4033-bc34-e9d8c9d79560 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:44.307343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.308173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.308450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.308732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.314501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 399/796] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:44.314501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.314501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.314501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.324046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:44.327542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:44.328770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-35bf5a6c-d4b0-4c54-9ef3-c43f72f7840e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:44.331594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:44.332796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:44.333625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 398/797] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:44.334304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.334304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.334630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.356306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:44.357583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:44.369289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.410287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 400/798] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:44.411265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.440139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 399/799] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:44.441393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.481065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.533935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28c19c-a44f-406d-967d-d831f8142ea9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.559418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a0657-97b7-4959-a8cc-c1a4bd32f6e5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.572361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.573257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.573547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.573829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/f0bf046435a84f7592fd022c03eda0a1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.575812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=f0bf046435a84f7592fd022c03eda0a1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:44.577285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:44.599846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 401/800] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:44 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/f0bf046435a84f7592fd022c03eda0a1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:44.599846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.607010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.607972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.608391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.608776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.609738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:44.610157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:44.611905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:44.627324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 400/801] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:44.628611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.635246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.737956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0dff11-866f-4cc4-8e26-9d530007fb3e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.756332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.761104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483eaeb-d8ca-4825-9687-e79e4aa3c7d0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.783131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.784301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.784756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.785279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.786163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:44.787790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:44.801674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.802543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.802833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.803115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:44.803744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:44.805246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 13:42:44.806434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:44.809048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 402/802] 10.222.0.24 () {56 vars in 1129 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:44.810046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.825259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:44.834073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:44.835025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:44.836439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 401/803] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:44.837548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:44.920330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:44.932351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266bf7e-6f65-4ba0-897b-c5fc8ef09859 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.967753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbdd641-aceb-418b-9ba7-ec0889d9f647 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:44.983564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:44.984617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:44.985093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:44.985442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:44.999774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.001545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.006743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.008179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.009017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.009498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.010410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.010937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.012491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.017648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 402/804] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.018684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:45.020448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:45.021309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:45.022184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:45.022184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:45.022184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:45.022184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.022184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 403/805] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:45.028326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.048193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.137089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1c463-1df7-48db-a81f-482c121c4ecd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.159418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b05f4-7e30-4f9f-90b0-20e6a3ebf577 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.174455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.175463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.175994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.176470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.177415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.177940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.179505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.188740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 403/806] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.189822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.193619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.194742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.194742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.194931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.196094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.196615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.198160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.206970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 404/807] 10.222.0.24 () {56 vars in 1124 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.207848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.221263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.305435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.322222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7730c16-2b33-4a9f-9ec3-f50ef8d8c6ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.325474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f9033-ae4d-4479-804e-772d21adacd6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.357172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.357973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.358266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.358548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.365041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.365956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.365956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.366252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.366981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.367454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.368867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.370058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.371569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.380907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 404/808] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.381699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4622d738-fda9-48fc-9ae6-4dbce375a983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:45.394249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:45.395325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:45.396443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:45.396443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:45.396443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:45.396443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.400219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 405/809] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:45.401380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d03f-de65-447b-891f-2da542d42499 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.401682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d03f-de65-447b-891f-2da542d42499 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.401973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471d03f-de65-447b-891f-2da542d42499 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.403383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 406/810] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:45.404051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471d03f-de65-447b-891f-2da542d42499 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.467170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.508271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd748c58-f99a-4b3e-b66a-4a02cc570e26 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.517937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471d03f-de65-447b-891f-2da542d42499 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.546278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.547499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.548123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.548667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.550067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.552436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536392d-ea66-4c76-b7f5-3f6579ef47cd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.556443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.557178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.557463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.557739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.565796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.567261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.571129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:45.582879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:45.583902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.584901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 407/811] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:45.585464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 405/812] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:42:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:45.585552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.586449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.586798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.587182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.621361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:45.623442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:45.701706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94b1c6-83c1-4012-9eb5-4b87e950a6ae None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.720538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.726049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 406/813] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:45.727317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.762825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.763544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.763844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.764192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.765060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.765491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.766898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.781529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 408/814] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.782343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.870987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12853b-7a7e-4d3e-929f-ebf699eccb35 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.899715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:45.909287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.910066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.910422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.910767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.911682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:45.912086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.913581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.919097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece3eb7-a55e-47a0-85ad-4901ad6bb41b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:45.935868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 407/815] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:45.935868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:45.961040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:45.962477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:45.962913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:45.963331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:45.964179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:45.965903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:45.979885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:45.980961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 13:42:45.981809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:45.984736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 409/816] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/services/network => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:45.985822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.042940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.049977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783c749-7390-40c7-8a1b-ccf61e6f957b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.085272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.086231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.086568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.086905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.094286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.095799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:46.111556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:46.112546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:46.113451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 408/817] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:46.114050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.121539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445689b-e2f6-4eeb-9c08-7dfbba6c22f4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.163117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.165004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.165722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.166068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.166405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.222881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552e3a5-12ce-49ba-a20d-e060f6635964 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.245422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.246894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.260413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.261274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.261626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.261972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.262820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] RBAC: Adding query filter params (name=network) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:46.263491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.265301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.271524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 409/818] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:46.272375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.316229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.364255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 410/819] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:42:46.365364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.391737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a58d90-eae4-4365-aa46-2c393899b106 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.401295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.428061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.428668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.428950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.429227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.430678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:46.430943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.432399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.450338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 410/820] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:46.451185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.476014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b2a047-d06f-4c33-95a9-5fa864a86653 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.517895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.518377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.518804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.519760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:46.520411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.522045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.533293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 411/821] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:46.534370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.544175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.564776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb911e4f-0a9e-49d1-b1f9-6318e851731b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.605889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.606681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.606964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.607241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.617817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.619255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:46.638126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:46.639226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:46.640286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:46.640286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:46.640286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:46.640286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:46.643989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 411/822] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:46.644752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.664036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.668263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33110725-edcf-4807-902c-8eccfed823e2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.704762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.705925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.706360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.706784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.719166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.720846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:46.741339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:46.742400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:46.743350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:46.743350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:46.743350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:46.743350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:46.745432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 412/823] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:46.746378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.752296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace8c04-7517-47ad-a05d-f009f2296a6c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.791389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.792086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.792378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.792661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.793567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:46.794954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:46.809916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.823865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 412/824] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:42:46 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:46.823865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:46.918245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b5348-a385-4778-915d-6eb753a505e7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.924283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:46.965768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d3a9e-f3af-4cc9-a0de-7d123f65be97 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:46.996431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:46.997183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:46.997981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:46.998282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:46.999032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:46.999534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.003841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.016594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 413/825] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.025471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.048801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.049484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.049771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.050052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.050801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:47.051115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.052534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc2967-24bf-4a76-ad01-0691cdddd671 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.073643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 413/826] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.075290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfdc0-39cc-45df-8396-05fb30ccca0f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.075809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfdc0-39cc-45df-8396-05fb30ccca0f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.076338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cfdc0-39cc-45df-8396-05fb30ccca0f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.078029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 414/827] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:47.080375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cfdc0-39cc-45df-8396-05fb30ccca0f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.170407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6b756-1434-4907-bb7e-352b4f0ed34f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.194894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.205965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cfdc0-39cc-45df-8396-05fb30ccca0f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.210357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df628868-c760-4791-839b-842dc6a84536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.211995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628868-c760-4791-839b-842dc6a84536 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.212416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628868-c760-4791-839b-842dc6a84536 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.212820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df628868-c760-4791-839b-842dc6a84536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.222833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df628868-c760-4791-839b-842dc6a84536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.225051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df628868-c760-4791-839b-842dc6a84536 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.243141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-df628868-c760-4791-839b-842dc6a84536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:47.243370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:47.244246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:47.245098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 414/828] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:47.246632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.247302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.247544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.253424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.254217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.254510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.254786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.263601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.265510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.271238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:47.272979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:47.281348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:47.282222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:47.283109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:47.283523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 415/829] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:47.284326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.360148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 415/830] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:47.361079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.389815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64ce8-5473-45f1-99e2-a54963f5a9d9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.425594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.426181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.426464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.426745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.427530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:47.429652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.429652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.444217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 416/831] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.444920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.479195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e5a54-8228-4c41-82fb-70a1d3eb86f6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.488465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.509719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.515859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.516537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.516823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.517104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.517893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:47.518203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.519635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.533138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 416/832] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.533958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.577764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2656f5c-0adb-435f-881c-50c5e6588d3f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.612795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.613583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.613867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.614141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.624080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:47.631655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:47.633490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 417/833] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:47.634589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.657611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9c68-63f2-4629-9586-e4a7ceef3166 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.693983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.694673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.695066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.695398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.696485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:47.696803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.698248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.712674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 417/834] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.713432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:47.752374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f37ff6-4e27-4728-8f9a-5817e1f0db55 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.761425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.771288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:47.809607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.810542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.810828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.811104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/6d80ba86a9bf4219a509d11f4617930a/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.838565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=6d80ba86a9bf4219a509d11f4617930a, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.841259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.863788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 418/835] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:42:47 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/6d80ba86a9bf4219a509d11f4617930a/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:42:47.866096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76c8f3-fa2e-40d6-86c7-bc1d961910b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:47.915020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:47.916174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:47.916573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:47.916854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:47.917691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:47.918007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:47.919445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f672dc-f4b7-4128-8a7d-e591a6ae68aa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:47.930429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 418/836] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:47.935897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.010497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7dd74-6629-4cab-aafc-bb8b6a95c60b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.011173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7dd74-6629-4cab-aafc-bb8b6a95c60b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.011653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7dd74-6629-4cab-aafc-bb8b6a95c60b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.013451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 419/837] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:48.020772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.021444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.021727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.043145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:48.044500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:48.058435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce0b36-3991-429a-8847-631fda7f221e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.105972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.106769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.107054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.107328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.118487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.120026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:48.139470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:48.140589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:48.142540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:48.142540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:48.142540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:48.142540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:48.142540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 419/838] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:48.143802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 420/839] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:48.146245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.161839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.167673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.197246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.272896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853555-7564-40cd-a130-6beec4c2c235 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.303427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0aa62c-12a3-46a0-a4e8-b2c0796d9d1b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.313370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.314321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.314817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.315299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.316316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:48.316833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.318424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.331504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 420/840] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:48.336953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.345652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.346488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.346770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.347044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.347709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.352838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:48.364621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:48.365804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:48.366757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:48.369746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 421/841] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:48.375532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.423895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.457677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62cc665-29ff-42a1-bd0c-ae3fc972a7d6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.472397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.486094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba593d8-528d-492c-b3b9-5abe739697d0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.505317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.506424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.506707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.508133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.519164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.521073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.535403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.536082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.536370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.536652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.537412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:48.537723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.539295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:48.539629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:48.540552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 13:42:48.541477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:48.548211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 422/842] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:48.548767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 421/843] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:48.550137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.550418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.666900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e77d2d-6de6-4796-885f-1ba12c210b91 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.688448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9409e1d-85e7-4862-a658-0225239cbef1 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.708908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.709924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.710434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.710911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.711843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:48.712377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.713948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.719327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 423/844] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:48.739618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.740793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.742857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.743556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:48.744982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.745519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:48.746112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:48.747943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:48.769705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 422/845] 10.222.0.24 () {56 vars in 1135 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:48.776242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:48.797927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:48.888289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e895fee-fb95-40c0-910e-8a50386561a8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:48.925091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:48.926153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:48.926460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:48.926746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.003801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.005287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.024187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:49.116228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 424/846] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:42:49.117861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:49.224545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547fedc8-ebd1-4ad3-93c8-4aa1d82306d8 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:49.260513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e494305-c882-464c-881f-47459b4a38da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:49.261311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494305-c882-464c-881f-47459b4a38da None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:49.261593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494305-c882-464c-881f-47459b4a38da None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:49.261868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e494305-c882-464c-881f-47459b4a38da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.272185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e494305-c882-464c-881f-47459b4a38da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.274244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e494305-c882-464c-881f-47459b4a38da None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9e494305-c882-464c-881f-47459b4a38da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:49.293041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:49.293939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:49.295030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:49.295030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:49.295030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:49.295030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:49.295030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 423/847] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:49.299765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:49.401621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27bd08-d2a9-442d-a06f-f0cafa8c1fed None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:49.428179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:49.443072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:49.444177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:49.444654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:49.445398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.446484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:49.446984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.448710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.465932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 425/848] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:49.471466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e494305-c882-464c-881f-47459b4a38da None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:49.524010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:49.581659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e494305-c882-464c-881f-47459b4a38da None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:49.617920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:49.618893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:49.619176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:49.619449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.626597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.628065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:49.642177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:49.643081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:49.644155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 424/849] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:49.649274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:49.760738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253da54-2640-4483-9191-b2142dfcb2fe None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:49.794929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:49.795605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:49.795917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:49.796202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.796995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:49.797310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.798751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d27ab7-d073-45ef-8091-75b4c51154d9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.811384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 426/850] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:49.817245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:49.923532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5a5cf-a31f-4811-bf5d-1a092dba69fb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:49.925302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:49.966322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:49.966590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:49.966841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:49.967035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:49.967945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=f0bf046435a84f7592fd022c03eda0a1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:49.968269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:49.969773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e910f3-c1b2-4bb4-a5a0-b69fb5c4db02 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:49.977756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 425/851] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:49 2022] GET /identity/v3/role_assignments?user.id=f0bf046435a84f7592fd022c03eda0a1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:50.165796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18876f-9206-4611-b55f-477b03d6cdd5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.166134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18876f-9206-4611-b55f-477b03d6cdd5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.166347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18876f-9206-4611-b55f-477b03d6cdd5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.167842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 427/852] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:50.173341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.173870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.174373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.192675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:50.194099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:50.196599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1466afc-e67a-4f7a-82b9-0c9ca9189591 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.196885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1466afc-e67a-4f7a-82b9-0c9ca9189591 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.197161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1466afc-e67a-4f7a-82b9-0c9ca9189591 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.198661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 428/853] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:50.203476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e657d-c643-47e5-b982-105a7077608c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.203767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e657d-c643-47e5-b982-105a7077608c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.204081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e657d-c643-47e5-b982-105a7077608c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.223181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-781e657d-c643-47e5-b982-105a7077608c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:50.224378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-781e657d-c643-47e5-b982-105a7077608c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:50.259466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:50.281269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 426/854] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:50.304674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.306152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 429/855] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:42:50.328408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e657d-c643-47e5-b982-105a7077608c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.415320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:50.422945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bc450b-a183-4176-8b3d-ebdd8e87c48b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.430975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e657d-c643-47e5-b982-105a7077608c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.463153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.464633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.465363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.466168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.467196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.467489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:50.468310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.468692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.469065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.469554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:50.479993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:50.483917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:50.484832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:50.485664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:50.488203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:50.489160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:42:50.489980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:50.496011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 430/856] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:50.496378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 427/857] 10.222.0.24 () {56 vars in 1132 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:50.497862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.498733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.531677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:50.615312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c69b-166b-4f3e-bfb5-579aace93a31 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.616944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7b65d5-58c6-4a8a-8287-909b01d1ade5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.659535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.660348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.660535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.661020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.661020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.661786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:50.662165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:50.662361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.662833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.663288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.664322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:50.665177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:50.665677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:50.667409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:50.674990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 428/858] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:50.676767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.679094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 431/859] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:50.684808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:50.713550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:50.787803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:50.803447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb17a8-bb0b-4d5d-acb5-c29b608cc170 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.813444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05abae6-ddb0-42a3-a8e2-db63f6bbfea6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:50.848324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.850202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.850758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.851580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.857183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:50.857936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:50.858230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:50.858513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:50.859253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:50.859567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:50.861030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:50.869724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:50.871679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:50.872006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 429/860] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d1459e10-7f67-4d49-84d0-9cb98dd223d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:50.893179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:50.894125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:50.894974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:50.894974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:50.894974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:50.894974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:50.897435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 432/861] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:50.904174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.007271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:51.021189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecd4aad-5783-4697-96b9-724b5d67efca None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.059867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.060537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.060821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.061098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.061830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:51.062136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.063494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:51.066558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8fe7-71f3-41cb-8933-7c53a85dba4d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.067296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8fe7-71f3-41cb-8933-7c53a85dba4d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.067296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fac8fe7-71f3-41cb-8933-7c53a85dba4d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.069567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 433/862] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:51.077086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.077420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.077520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 430/863] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:51.077803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.082856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.098153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:51.099349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:51.174161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 434/864] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:51.197697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.197890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b12f9a-c6f5-4453-96b4-3063a1b5c3f0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.235715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.236681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.237004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.237280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.245743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.247186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:51.266077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:51.267581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:51.268545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 431/865] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:42:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:51.277597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:51.278675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.293692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:51.323062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ae0a4-47f1-422f-9089-b1b53bbfc816 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.360076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.361045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.361512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.362017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.363019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.364699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:51.381736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 435/866] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:42:51 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:51.410900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7545e3-ed2c-4ead-b8a7-d0a48b92f0d7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.457652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.458345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.458633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.458916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.459783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:51.460134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.461909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:51.483033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 432/867] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:42:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:51.488454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.610250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb5d04-03bc-41a0-bf90-ddff4fde712a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.638770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:51.645316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.646161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.646664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.647105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.722670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.724195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:51.828568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 436/868] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:51 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:42:51.837107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:51.902603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:51.943706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320cefe3-67c5-4464-a931-ac685995a5b0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:51.980669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:51.981470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:51.981758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:51.982040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:51.992397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:51.993838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:52.012587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:52.013447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:52.014301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:52.014301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:52.014301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:52.014301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:52.014301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 433/869] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:52.020214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.051605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.127218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee044fda-ac67-4599-b442-ceb438fae396 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.173880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.174476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.174768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.175053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.175796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:52.176143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.177528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915429e-9e96-4854-af89-62d020f6c47d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.189822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 437/870] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:52.195750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.312077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d28af-497d-4078-b542-b418edd66bb5 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.326862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.330380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f588-13a2-49a3-b3a1-2b5b59e0095c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.330701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f588-13a2-49a3-b3a1-2b5b59e0095c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.331016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39f588-13a2-49a3-b3a1-2b5b59e0095c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.332696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 438/871] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:52.338835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.339107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.339389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.354452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.355566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.356098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.356587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.357526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:52.358826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:52.365230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.367578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:52.386590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:52.387473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:52.388378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:52.389523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 434/872] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:52.394408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.447860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 439/873] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:42:52.467685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.490402 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.520563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13317f7a-b8f0-4e2a-b677-8e83122de827 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.556433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.557356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.557831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.558292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.559258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:52.559749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.561451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.574886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 435/874] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:52.582891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d86f6ec-5d3f-4e8d-ab78-5a89d069267a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.583042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.615513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.616519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.616907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.617593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.618387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.619899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.621028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:52.636070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:52.637098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:52.637919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 440/875] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:52.643211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.707099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd16932-986f-48bf-9934-fc49bd09201f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.738228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.755852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.756317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de3528-bd40-4b74-b239-9b21561c5a74 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.756552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.756847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.757965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.759082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=6d80ba86a9bf4219a509d11f4617930a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:52.759411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.761869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.773948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 436/876] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/role_assignments?user.id=6d80ba86a9bf4219a509d11f4617930a&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:52.806854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.807981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.808504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.808995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.809955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:52.810535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.812329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32429386-e402-4596-b46b-38bd22a90bee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.818604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 441/877] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:52.825876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:52.937360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:52.941871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0808e5b-0142-41b8-95d3-7c85dce9e40e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:52.985397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:52.986004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:52.986294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:52.986575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:52.987302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:52.987617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:52.989021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60893b2-22f8-4c12-a531-7cd7f933fb02 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:52.994482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 437/878] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:42:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:53.302414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2354-5318-495e-8188-fe70652f0fa8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:53.302750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2354-5318-495e-8188-fe70652f0fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:53.303110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf2354-5318-495e-8188-fe70652f0fa8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:53.305396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 442/879] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:53.310720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddeb5f-abfc-4a0d-b291-725c651a0a55 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:53.311014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddeb5f-abfc-4a0d-b291-725c651a0a55 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:53.311294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddeb5f-abfc-4a0d-b291-725c651a0a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:53.328916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-01ddeb5f-abfc-4a0d-b291-725c651a0a55 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:53.330105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-01ddeb5f-abfc-4a0d-b291-725c651a0a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:53.401735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 438/880] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:42:53.421418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf2354-5318-495e-8188-fe70652f0fa8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:53.537750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf2354-5318-495e-8188-fe70652f0fa8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:53.574119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:53.574704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:53.574988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:53.575271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:53.576022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:53.577392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02372b3-517b-4ad4-9607-477726ce245a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:53.594006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 443/881] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:42:53 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:53.863325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7be80-d2b9-4d8c-8567-764d8542d99d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:53.864034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7be80-d2b9-4d8c-8567-764d8542d99d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:53.864544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7be80-d2b9-4d8c-8567-764d8542d99d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:53.866375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 439/882] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:53.872302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:53.873168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:53.873655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:53.891441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:53.892971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:53.895189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:53.972881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 444/883] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:53.995305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7be80-d2b9-4d8c-8567-764d8542d99d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:54.093763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:54.103808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7be80-d2b9-4d8c-8567-764d8542d99d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:54.139854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:54.140590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:54.140876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:54.141157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:54.141819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:54.143200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:54.156127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:54.157184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:54.158018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 440/884] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:54.164524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:54.270543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1ac4b-3c34-41cb-9914-bd3f714f25e7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:54.306612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:54.307292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:54.307577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:54.307878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:54.308592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:54.308898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:54.310270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:54.312175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:54.315975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 445/885] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:54.321955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:54.423018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7c316-a25a-4d5c-b25d-8c6f142e3a68 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:54.457119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:54.457781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:54.458067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:54.458348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:54.459058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:54.459371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:54.460977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:54.466188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 441/886] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:54.472024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:54.591065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb03725-f92c-45d8-87f0-aaff778d22a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:54.626885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:54.627547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:54.627851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:54.628134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:54.628929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] RBAC: Adding query filter params (interface=public, service_id=b931635c0d6c48d18eb9e54bdd62994e, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:54.629249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:54.630618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d291a9a-e76f-4028-a26e-bf6dd842a016 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:54.636537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 446/887] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:42:54 2022] GET /identity/v3/endpoints?service_id=b931635c0d6c48d18eb9e54bdd62994e&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:54.641885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:54.665460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:54.746184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccef0b9-5ab4-40f5-8b9d-fdd4e195ea12 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:54.781093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:54.781761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:54.782044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:54.782323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:54.783016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:54.784424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daec8e-d47e-4918-822f-999970bb10e7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:54.791727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 442/888] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:42:54 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:55.287104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94a36-8b8d-4bd3-8f0f-5cd8b0bf714a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.287875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94a36-8b8d-4bd3-8f0f-5cd8b0bf714a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.287875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e94a36-8b8d-4bd3-8f0f-5cd8b0bf714a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.289742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 447/889] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:55.295261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e31b-64a2-4650-8af2-aaa4871c5202 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.295681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e31b-64a2-4650-8af2-aaa4871c5202 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.296108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6e31b-64a2-4650-8af2-aaa4871c5202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.320246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-caa6e31b-64a2-4650-8af2-aaa4871c5202 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:55.321449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-caa6e31b-64a2-4650-8af2-aaa4871c5202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:55.389371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 443/890] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:55.413843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94a36-8b8d-4bd3-8f0f-5cd8b0bf714a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:55.481912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:55.537011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e94a36-8b8d-4bd3-8f0f-5cd8b0bf714a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:55.572972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:55.573565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.573857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.574142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.574873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:55.576288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76303f39-80cd-48d0-a235-3a74258331cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:55.595990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 448/891] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:42:55 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:42:55.664832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6b03-f67e-4f7b-93dd-2fed09385830 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.665837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6b03-f67e-4f7b-93dd-2fed09385830 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.666330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a6b03-f67e-4f7b-93dd-2fed09385830 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.668340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 444/892] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:55.674399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.674703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.675036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.694697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:55.696030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:55.780539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 449/893] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:55.805366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6b03-f67e-4f7b-93dd-2fed09385830 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:55.822792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:55.915556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a6b03-f67e-4f7b-93dd-2fed09385830 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:55.951803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:55.953064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:55.953552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:55.954041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:55.967940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:42:55.977204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:55.978266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:42:55.979140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:55.981718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 445/894] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:42:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:55.987896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.042725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.099997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3db04d-7b6b-45ef-9afe-2d9d038e7da6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.138958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.139925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.140395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.140828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.141741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:56.142213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.143807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.153260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 450/895] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:42:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:56.158945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.249094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.270023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a868191-3f54-44e6-9ca5-41acb522af18 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.305881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.306678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.306965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.307239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.319104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.320618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ad874e33-9228-4eab-b75f-e19948cda473 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:56.338613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:56.339546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:56.340454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:56.340454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:56.340454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:56.340454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:56.340454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 446/896] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:56.349038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.433335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700a65-11cc-43f2-8a90-4a79fe2b2555 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.433680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700a65-11cc-43f2-8a90-4a79fe2b2555 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.433908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed700a65-11cc-43f2-8a90-4a79fe2b2555 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.436601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 447/897] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:42:56.440549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.440843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.441129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.459705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:56.461285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:56.462138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec990c1-c06c-482f-bd25-5f43c7ef98f9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.473471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.498599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.499449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.499892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.500060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.503696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:56.503696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.503696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.517468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 451/898] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:56.522629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.552093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 448/899] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:42:56.575412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.637194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.641017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae5e15-e68a-4f3a-b997-3d0e1736e21d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.690321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.691640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.692160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.692986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.697023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7760e-3649-4b4e-8d44-36fa2f280b4e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.702705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.704354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:42:56.723359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:56.724604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:42:56.726324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:56.734356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 452/900] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:56.735797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.760294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.761038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.761324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.761600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.762263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.763632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.775388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:56.776786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:56.778052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:42:56.779053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:56.783089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 449/901] 10.222.0.24 () {56 vars in 1132 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:56.784358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.856013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38689786-4af2-4ccd-bd99-0bc02e06fc12 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.894721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.895431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.895717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.896027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.896781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090d0f3-5ab9-4d24-afaf-41a613540434 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:56.896897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:56.897199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.899639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.920518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 453/902] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:56.922136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:56.932133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:56.939289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:56.940724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:56.941198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:56.942077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:56.942951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:56.943810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:56.946020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:56.958086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 450/903] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:56.960056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.049594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.067000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bca853-00fe-49e4-8f21-b6a808ea36dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.104002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.104661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.104952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.105234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.106448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ad509-24cd-47ea-a739-7fa5aedad3a2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.146345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.147017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.147307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.147912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.148516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:57.148833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.150203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5321cf-edbc-463f-bc11-8b1c5295997e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.159424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 451/904] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:57.159424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f844-567d-46cd-b66f-23203e4218e9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.159424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f844-567d-46cd-b66f-23203e4218e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.159424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881f844-567d-46cd-b66f-23203e4218e9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.159424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 452/905] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:57.161875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881f844-567d-46cd-b66f-23203e4218e9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.183842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.185454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfc82e-99d0-491b-bae9-649d69c877ce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.215910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.271704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881f844-567d-46cd-b66f-23203e4218e9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.279363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.297436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 454/906] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:42:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:42:57.298991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.299354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.299765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.309714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.310383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.310670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.310951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.311752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] RBAC: Adding query filter params (interface=public, service_id=512d5e123134412c91b76029fa800f39, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:57.312108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.313475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.317020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:57.318231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:57.321836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 453/907] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/endpoints?service_id=512d5e123134412c91b76029fa800f39&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:57.322836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.396433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 455/908] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:57.398021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.439770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c7ad7-9667-44cc-9d0d-e509110189af None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.454990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.478377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.479175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.479465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.479749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.490862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.495224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.515759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a237e2-cf97-45ea-b962-849367463e74 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:42:57.519422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:57.520331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 454/909] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:57.526709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.538781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.557856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.558578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.558936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.559293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.560081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.561543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.568440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 456/910] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:57.569223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.648306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0ae44-230a-4d9e-942a-e22c81132966 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.693429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.694271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.694596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.694917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.696032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.697539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:57.711709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:57.712855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 13:42:57.713687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:42:57.718171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 455/911] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:57.719151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.724709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe231e1a-4870-47f0-8126-863302d83976 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.758702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.765742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.766419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.766708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.767732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.768976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:57.768976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.770209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.785507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 457/912] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:57.792075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.847132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:57.852790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0cff1e-25b7-467f-96f6-f64d8173b393 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.888579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.889256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.889547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.889828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.890605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:57.890917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.892375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.897772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 456/913] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:57.903393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:57.906473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d38690-59ed-477a-8c05-749be057150f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:57.960392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:57.962149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:57.962689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:57.963178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:57.973620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:57.976806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-23f2b694-b706-49d6-b32f-788b56bad88e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:42:57.997073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:57.998038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:57.998914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 458/914] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:42:58.000063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e02d4-44d9-49ee-a3e3-95100742f42e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.000167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e02d4-44d9-49ee-a3e3-95100742f42e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.000387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e02d4-44d9-49ee-a3e3-95100742f42e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.001769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 459/915] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:58.004522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e02d4-44d9-49ee-a3e3-95100742f42e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.015150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.025563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb74bbb-1fbe-4ffd-8ade-02813e4a6a44 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.061655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.062332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.062625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.062912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.063631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.063980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.065361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5ca86-42ff-4df4-9939-79607a8538b0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.070828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 457/916] 10.222.0.24 () {56 vars in 1114 bytes} [Tue May 24 13:42:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:58.072031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.072329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.072614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.089291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:58.090484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:58.124158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e02d4-44d9-49ee-a3e3-95100742f42e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.126330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.162438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 458/917] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:42:58.163876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.169391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.170295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.170585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.170863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.171658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.172004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.173446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.189716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 460/918] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:58.190845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.263443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.280341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaaf7d-60d6-41a2-ad30-de829e23766b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.301693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea210e5-d97c-4a2e-984a-ec429091116e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.321300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.321968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.322254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.322537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.323286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.324680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.341374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.342435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.342940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.343170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 459/919] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:42:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:42:58.343566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.344009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.344614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.345996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:42:58.370241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:42:58.371341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:42:58.372505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 461/920] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:42:58.377849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.420346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.521293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b5b95-1a40-4b97-ae68-a15c237f9196 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.532010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2810a5-8bba-4d02-80fc-57930e086b2f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.567101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.568111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.568592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.569050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.570467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] RBAC: Adding query filter params (role.id=ca62ce8eb7e94f379ae605ecd85a3683, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=6d80ba86a9bf4219a509d11f4617930a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.570999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.572459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.573312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.574821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.575288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.575680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.576508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.576822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.578192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.578531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 460/921] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/role_assignments?user.id=6d80ba86a9bf4219a509d11f4617930a&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=ca62ce8eb7e94f379ae605ecd85a3683 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:58.584735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 462/922] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:42:58.591498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.594528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.708815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d81f05-bdc0-4f36-ada4-e5eb5ec384dd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.748660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.749801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.750329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.750836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.751860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.752406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.754126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.761696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 461/923] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:58.768944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:58.872272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:58.882917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea534a9-df1f-4242-af6d-caa66fa78b25 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:58.917429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:58.918141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:58.918431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:58.918709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:58.919506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] RBAC: Adding query filter params (interface=public, service_id=42ce7b8d66564615aacfbe9625c5e0e1, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:42:58.919836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:58.921209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651db207-235e-4d96-be55-c168a0a3a8b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:58.926950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 463/924] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/endpoints?service_id=42ce7b8d66564615aacfbe9625c5e0e1&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:42:58.932179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:42:59.000915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:42:59.039697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ea4b7-e628-4673-80a0-9080d8bc6a32 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:42:59.074207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:42:59.074887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:59.075182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:59.075462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:59.076187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:42:59.077566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd5cdd-cbbe-4c8c-9650-c5c34b8841f3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:42:59.087332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 462/925] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:42:58 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:42:59.885592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a29-2b0e-4eb8-8ecf-17ea973a1f35 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:59.886567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a29-2b0e-4eb8-8ecf-17ea973a1f35 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:59.887083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65286a29-2b0e-4eb8-8ecf-17ea973a1f35 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:59.888910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 464/926] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:42:59.894552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:42:59.895352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:42:59.895863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:42:59.913666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:42:59.915189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:42:59.982232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 463/927] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:42:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:00.005452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286a29-2b0e-4eb8-8ecf-17ea973a1f35 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:00.112499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65286a29-2b0e-4eb8-8ecf-17ea973a1f35 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:00.148092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:00.148845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.149134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.149414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.150080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:00.151459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:00.164635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:00.165594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:00.166409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 465/928] 10.222.0.24 () {56 vars in 1132 bytes} [Tue May 24 13:43:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:00.173871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:00.238961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:00.282355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238b962-f950-4fdc-a95f-92e0e1a3b7cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:00.318530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:00.319208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.319531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.319835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.320564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:00.320878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:00.322251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:00.327806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 464/929] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:43:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:00.333371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:00.360320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:00.446423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0f2e2-0a66-4da6-af0e-86d7edcc7f54 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:00.480812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:00.481483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.481771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.482050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.482769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:00.483081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:00.484605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7023ab16-f924-4a04-8733-f9df81b4c1b6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:00.490159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 466/930] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:43:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:00.495801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:00.608391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed5dd7-6fe5-4bdb-b805-6e8c5e0328f7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:00.642992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:00.646975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:00.648274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.648758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.648920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.649603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:00.651048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae0fec-c8fa-4211-8198-6ebcfe1537b3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:00.677456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 465/931] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:43:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:00.805098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be89788-413a-4b01-8b31-656f3ab15662 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.805425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be89788-413a-4b01-8b31-656f3ab15662 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.805707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be89788-413a-4b01-8b31-656f3ab15662 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.807424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 467/932] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:00.813333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.813636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.813920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.833975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:00.835334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:00.859295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c378aa-8e8d-49dc-b36d-9f957a38af72 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.859621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c378aa-8e8d-49dc-b36d-9f957a38af72 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.859900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c378aa-8e8d-49dc-b36d-9f957a38af72 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.861516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 468/933] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:00.867126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:00.867429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:00.867688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:00.891235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:00.892467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:00.921415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 466/934] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:00.945525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:00.978686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 469/935] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:01.000913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.021981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.073113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb84421-6f79-4f5f-9f7f-88f5ebede899 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.080777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.110522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.111338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.111627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.111939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.112658 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.114192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.115680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING keystone.server.flask.common [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 13:43:01.134572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75d9f31-08cb-4557-bf03-37274d9a1d2c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.137665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 13:43:01.137665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 13:43:01.137665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [parameters: {'id': '1e62b4c2a3f04bf7a7572466ad26ffc4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 13:43:01.137665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1e62b4c2a3f04bf7a7572466ad26ffc4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1e62b4c2a3f04bf7a7572466ad26ffc4. May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:43:01.146390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 24 13:43:01.147244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return project_ref.to_dict() May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 24 13:43:01.148374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 13:43:01.149251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 13:43:01.150483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:43:01.151574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'id': '1e62b4c2a3f04bf7a7572466ad26ffc4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.153446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:01.155120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict( May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1e62b4c2a3f04bf7a7572466ad26ffc4. May 24 13:43:01.156967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.159961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 467/936] 10.222.0.24 () {60 vars in 1140 bytes} [Tue May 24 13:43:00 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:01.168558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.179447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.180559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.181030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.181510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.196295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:01.212125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:01.213122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:01.213934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:01.221412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 470/937] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:43:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:01.222431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.302769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734de4c-53bf-4bf8-a638-d7ff5e308f97 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.315741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.352454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a86b7-557c-4908-9443-43acf81c1db1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.355736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.356823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.357335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.357827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.367072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.369600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:01.388054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:01.388955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:01.389823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:01.390229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.390229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.390229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.390229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.390494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:01.391084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.392119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.393092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 468/938] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:01.399065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.403545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 471/939] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:01.409295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.433665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.520146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecff34dc-e808-41ea-89cd-fa78630e2a29 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.530659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d2db5c-5fea-4e3e-b210-e130036d4083 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.559432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.560273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.560566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.560850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.563766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.567108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.567805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.568123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.568403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.569173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:01.569600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.571183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c72b29-5b97-4234-99bc-0faaf7dc5b64 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.574005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.575451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.593039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 469/940] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:01.594255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770806c1-0aef-40f1-975a-a861cbb4cc08 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.594557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770806c1-0aef-40f1-975a-a861cbb4cc08 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.594843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770806c1-0aef-40f1-975a-a861cbb4cc08 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6f69e635-2d96-4863-9cb8-f4c6a31cef10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:01.595112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:01.596039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:01.597078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:01.597078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:01.597078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:01.597078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:01.599134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 472/941] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:01.600123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 470/942] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:01.601761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.602060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.602346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.608729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770806c1-0aef-40f1-975a-a861cbb4cc08 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.626520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:01.628421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:01.711928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.735870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 473/943] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:01.745421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770806c1-0aef-40f1-975a-a861cbb4cc08 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.759324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.788766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.789858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.790356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.791648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.792689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:01.793222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.794860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.808825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 471/944] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:01.815679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.892312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1687e6-93de-4e67-a7bf-5e664871f0bc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.911296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:01.938184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.939016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.939442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.939731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.940527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:01.941928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:01.946743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f55832-b187-492c-b4b5-9e3c9da09a80 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:01.956557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:01.957867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:01.958712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 474/945] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:01.966511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:01.989455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:01.990288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:01.990697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:01.990910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:01.999410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.000919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:02.017554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:02.018549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:02.019500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 472/946] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:02.024957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.029186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:02.099746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c4db3-8743-4817-a559-e85861e2ee63 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.136450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.137315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c12e-f90a-4198-8d08-0b513ec80bce None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.137756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.138267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.138764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.139686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:02.140343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.141925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.152127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 475/947] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:02.154127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.174766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.175500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.175799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.176113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.177010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:02.177376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.178888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.197834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 473/948] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:02.203034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.203709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:02.277764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec4db3-e3b2-4795-9c79-cc6cc63f19d7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.295874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:02.316514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.317305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.317665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.318019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.318870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:02.319317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.321803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.334007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 476/949] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:02.335597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.361231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f67c780-dd5c-47fa-9d08-483b0cd19786 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.408808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.410540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.411027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.411753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.426493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.429357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:02.452905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:02.453862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:02.454745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:02.454745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:02.454745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:02.454745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:02.458924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 474/950] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:02.459926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.461560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56143c2d-21aa-41a7-8b61-65f596f788b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.474138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:02.514051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.514990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.515474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.516001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.516971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.518551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.542086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 477/951] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:43:02 2022] POST /identity/v3/endpoints => generated 349 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:02.585896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf1b55-5155-4367-acc5-71ea76e21a91 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.602779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:02.636414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.637507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.638005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.638482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.639465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:02.640088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.641703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.657343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 475/952] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:02.665427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.791395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dec37da-b562-4880-89c8-d4f373fc56b9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:02.827839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:02.828706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:02.829056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:02.829341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:02.838722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:02.840324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a4cdd6cb-4b11-43e5-a32e-7abcd0aeb26f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:02.858546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:02.859504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:02.860893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 478/953] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:02.866640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:02.988252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d15c1d-475f-4af1-b9f6-1771f03b0aab None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:03.026689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:03.027782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.028274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.028732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.029781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:03.031767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f59d2-b5f0-4db5-a5c5-afd911629474 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.032265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f59d2-b5f0-4db5-a5c5-afd911629474 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.032357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f59d2-b5f0-4db5-a5c5-afd911629474 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.033827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 479/954] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:03.033934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:03.035261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:03.039886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.040434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.040925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.049842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 476/955] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:03.056031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:03.059001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:03.060475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:03.079051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:03.087178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:03.141323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 480/956] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:03.160791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:03.163921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba77ccc-f60b-4801-a1af-5a199b3f0971 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:03.204814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:03.205852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.206596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.206983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/6d80ba86a9bf4219a509d11f4617930a/roles/ca62ce8eb7e94f379ae605ecd85a3683` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.210131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=6d80ba86a9bf4219a509d11f4617930a, role_id=ca62ce8eb7e94f379ae605ecd85a3683)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:03.212589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:03.231860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 477/957] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:43:03 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/6d80ba86a9bf4219a509d11f4617930a/roles/ca62ce8eb7e94f379ae605ecd85a3683 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:43:03.278419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f97ff-3fcc-497a-89a0-9115caf45242 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:03.317673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:03.318836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.320081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.320813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.321777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:03.323434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:03.337628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:03.338676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:03.339738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:03.343637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 481/958] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:03.352349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:03.450484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:03.463544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9bd1-9673-4ca9-97fb-7e3c09c99ff4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:03.509562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:03.510485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.511092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.511572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.512552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:03.513068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:03.514634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafdda8-47e2-45cb-a440-c727d5a8a16d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:03.520538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 478/959] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:03.527840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:03.546410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:03.635342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c9178-54e3-4674-b180-8b514329cea4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:03.670866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:03.671926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.672441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.672928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.673916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] RBAC: Adding query filter params (type=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:03.674468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:03.676058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635e1fa-385e-4e7c-aa2c-984d0ba01cae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:03.682038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 482/960] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:03.986100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b29b-33ab-47d7-8fa2-d7ca0187a2dc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.986400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b29b-33ab-47d7-8fa2-d7ca0187a2dc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.986587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852b29b-33ab-47d7-8fa2-d7ca0187a2dc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:03.988265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 479/961] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:03.993752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:03.994077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:03.994365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:04.012266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:04.013446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:04.091941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 483/962] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:04.109575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852b29b-33ab-47d7-8fa2-d7ca0187a2dc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:04.212515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852b29b-33ab-47d7-8fa2-d7ca0187a2dc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:04.248801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:04.249480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:04.249768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:04.250047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:04.250773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:04.252234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:04.266766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.sql.core [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 13:43:04.266766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 13:43:04.266766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [parameters: {'user_id': '05180341eed3435cab70713a27f5fd05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 13:43:04.266766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80166) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:43:04.275468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:43:04.276368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application yield resource May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application next(self.gen) May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.rollback() May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.force_reraise() May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise self.value May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application session.commit() May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 13:43:04.277802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._prepare_impl() May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.session.flush() May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._flush(objects) May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application compat.raise_( May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application flush_context.execute() May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rec.execute(self) May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 13:43:04.278887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = connection._execute_20( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 13:43:04.280245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result = self._query(query) May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application conn.query(q) May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application result.read() May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 13:43:04.282195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [parameters: {'user_id': '05180341eed3435cab70713a27f5fd05', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:04.284163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 24 13:43:04.286119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:04.293541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 480/963] 10.222.0.24 () {60 vars in 1131 bytes} [Tue May 24 13:43:04 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:04.298110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:04.311528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:04.404892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9ebfd-1f06-448e-8503-2e26f3b0dcf6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:04.443630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:04.444869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:04.445354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:04.445841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:04.453612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:04.455311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6a9e54f7-b8e1-4426-a4bc-f11ae7fbf436 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:04.470927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:04.471910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 13:43:04.472855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:04.479993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 484/964] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:04.481871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:04.505244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:04.590903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f6bf64-4c26-455f-b1b8-0b59415254bb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:04.624691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:04.625413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:04.625697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:04.626023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:04.626836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:04.627142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:04.628790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad343b-b45f-4f63-998e-c1f2f68a155d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:04.671000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 481/965] 10.222.0.24 () {56 vars in 1101 bytes} [Tue May 24 13:43:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:04.991919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1a33-5e42-4526-b8a3-d2080f65eba1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:04.992309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1a33-5e42-4526-b8a3-d2080f65eba1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:04.992510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb1a33-5e42-4526-b8a3-d2080f65eba1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:04.994011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 485/966] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:04.999224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:04.999693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:04.999887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.018423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:05.019973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:05.089666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 482/967] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:05.111709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb1a33-5e42-4526-b8a3-d2080f65eba1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.184225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.216595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb1a33-5e42-4526-b8a3-d2080f65eba1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.250766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.251501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.251794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.252100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.252767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:05.254139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:05.267222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:05.268247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:05.269073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:05.271884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 486/968] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:05.274617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.276826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.379832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99614a-a02e-4f82-9781-17ad8cabb4ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.415097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.415851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.416150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.416478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.417213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:05.417526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:05.419894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36361170-db1c-4355-86d9-854ba8b922a1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:05.431169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 483/969] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:05.432186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.542680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea921-f564-4ca2-bce0-2cd5155c0d5d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.542874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd932-57ea-45a9-aac7-d12f444ed5ac None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.542958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea921-f564-4ca2-bce0-2cd5155c0d5d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.543246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ea921-f564-4ca2-bce0-2cd5155c0d5d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.544787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 484/970] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:05.549542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.549833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.550121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.567590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:05.568901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:05.583546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.584240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.584533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.584918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.585638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:05.585954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:05.587336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:05.589781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.593839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 487/971] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:05.598995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.635494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.643647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 485/972] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:05.666986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.705994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c532-2849-4175-9b71-b79e0875960e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.745853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.746964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.747315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.748034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.749195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] RBAC: Adding query filter params (interface=admin, service_id=42ce7b8d66564615aacfbe9625c5e0e1, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:05.749518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:05.751604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:05.759229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 488/973] 10.222.0.24 () {56 vars in 1249 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/endpoints?service_id=42ce7b8d66564615aacfbe9625c5e0e1&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:05.764413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.772838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1b568-c150-463a-baff-4afde4a25306 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.815070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.815996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.816298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.816582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.828277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:05.831733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:05.837635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:05.838664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:05.839506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 486/974] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:05.844727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:05.876346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1057b0-21f4-4949-8042-99c23b5243fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.907757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:05.910724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.911398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.911689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.912035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.912831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:05.914382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1310b018-15e5-4b3e-9725-48ac6a5d312f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:05.920362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 489/975] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:43:05.960080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385472b-4dbc-435f-8cf4-4c0d07ceae11 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:05.994027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6d15-4875-4a60-89bd-b2c8ae9293ff None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:05.994333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6d15-4875-4a60-89bd-b2c8ae9293ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:05.994628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ce6d15-4875-4a60-89bd-b2c8ae9293ff None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:05.996178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 490/976] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:43:05.998869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:05.999565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.000170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.000258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.001012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:06.001332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.002248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.002616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.002852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.002961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.018133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 487/977] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:06.019585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.024867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:06.026098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:06.098292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 491/978] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:06.121992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.137450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e6e7-34cc-439c-b83f-9541140e4e14 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.158618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:06.181651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.182437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.182726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.183270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.186050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:06.195725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.197241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:06.215959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:06.216932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:06.217833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:06.217833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:06.217833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:06.217833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:06.217833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 488/979] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:06.222989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.235336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7650e6-c16c-401b-86d1-9d84ea9c1d3e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.271048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.271652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.271965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.272305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.273047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.274432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.289028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 492/980] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:43:06 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:43:06.339092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238c130-2823-40fd-a0a1-8212018b9505 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.384729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.385415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.385700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.385979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.386780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:06.387096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.388593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.404083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 489/981] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:06.408377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.523812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02f155-c60a-4434-85a0-3c2beca26136 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.542106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:06.561213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.562031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.562317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.562593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.571207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.572699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:06.587578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:06.588706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:06.589670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 493/982] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:43:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:06.594473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.623355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:06.698946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbfffa-d589-40c8-867b-328ecdd32ead None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.737897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.738578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.738866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.739147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.740030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:06.740349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.742346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:06.764848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 490/983] 10.222.0.24 () {56 vars in 1149 bytes} [Tue May 24 13:43:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:06.766275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:06.869568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554212e-2ef7-4655-a36d-7fd1b0b46765 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:06.905820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:06.906472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:06.906762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:06.907047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:06.952773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:06.976115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:06.977582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.082206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 494/984] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:06 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 13:43:07.083420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:07.172145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:07.188985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894e311-bd21-4269-afb0-c2ba8bd12b7a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:07.223593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:07.224429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:07.224719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:07.225000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:07.235291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:07.236776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:07.255213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:07.257048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:07.258046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:07.258046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:07.258046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:07.258046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:07.258046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 491/985] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:07.264060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:07.342443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:07.365606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ce659-d9a0-478d-82cc-3b4af8989b8e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:07.404125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:07.404791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:07.405081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:07.405361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:07.406080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:07.406391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:07.407760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.423660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 495/986] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:07.424829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:07.539502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05454b94-a190-412a-bcff-6ff5c4820327 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:07.573278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:07.574072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:07.574358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:07.574638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:07.577323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:07.582274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:07.583803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:07.599031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:07.599991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:07.605143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 492/987] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:07.608703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:07.712200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bc5a-01f8-4c50-881b-a9b540874059 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:07.746068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:07.746735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:07.747025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:07.747308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:07.748144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:07.748461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:07.749964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053aea-053e-41ad-85cc-48b64679d58f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.758029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:07.765516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 496/988] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:07.770478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:07.878578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc32e-c394-4ed5-8831-0260d161b476 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:07.911776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:07.912459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:07.913279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:07.913694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:07.914533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] RBAC: Adding query filter params (role.id=ca62ce8eb7e94f379ae605ecd85a3683, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=6d80ba86a9bf4219a509d11f4617930a) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:07.914851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:07.916305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d248459-b88a-4029-8ed0-7fb0d34773ff None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:07.922648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 493/989] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:07 2022] GET /identity/v3/role_assignments?user.id=6d80ba86a9bf4219a509d11f4617930a&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=ca62ce8eb7e94f379ae605ecd85a3683 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:08.205612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceb068-9502-46c1-800e-e383547f8f2d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.205997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceb068-9502-46c1-800e-e383547f8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.206179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ceb068-9502-46c1-800e-e383547f8f2d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.207668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 497/990] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:08.213345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.213783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.213963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.232954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:08.234138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:08.292569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:08.305553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 494/991] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:08.328036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ceb068-9502-46c1-800e-e383547f8f2d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.437221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ceb068-9502-46c1-800e-e383547f8f2d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:08.471495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:08.472254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.472542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.472820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.473467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:08.474872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f049f1b2-1858-44eb-9388-cfca66cfbd43 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:08.487361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:08.488588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 13:43:08.489408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.497974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 498/992] 10.222.0.24 () {56 vars in 1126 bytes} [Tue May 24 13:43:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:08.498131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.557500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5ccb-d5b2-443e-86b8-0ca53fa6799a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.557784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5ccb-d5b2-443e-86b8-0ca53fa6799a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.558062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e5ccb-d5b2-443e-86b8-0ca53fa6799a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.559418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 499/993] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:08.565027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.565319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.565594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.589011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:08.590578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:08.604501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ac0b5-25c3-4f98-8150-6a13844a1585 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:08.641514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:08.642988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.643489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.644020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.645304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:08.645825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:08.648000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:08.653519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 495/994] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:08.658691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.667171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:08.684348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 500/995] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:08.703643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.720993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:08.770919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ace397-733a-4a46-a4be-8ae879ae2f62 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:08.806155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:08.806914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.807300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.807605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.808501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:08.808814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:08.809911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c7def-1a76-4c6e-9519-0c137564da56 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:08.810179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:08.814912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 496/996] 10.222.0.24 () {56 vars in 1122 bytes} [Tue May 24 13:43:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:08.820527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.846901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:08.848040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.848384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.848736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.849395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:08.851554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:08.863666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:08.868025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:08.868913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 501/997] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:08.871039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:08.929663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:08.930011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194bfd6e-b973-490f-ac52-1715ad8b424e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:08.950028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:08.971020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:08.971791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:08.972126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:08.972529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:08.973280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:08.974566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:08.975243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4fbd1-61a6-4899-a7a8-0ee764a58be0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:09.000030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 497/998] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:43:08 2022] POST /identity/v3/endpoints => generated 325 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:43:09.017768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:09.018609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:09.018967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:09.019321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:09.020160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:09.020535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:09.023038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:09.029115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 502/999] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:09.034729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:09.151188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8429f6-6542-4990-a006-740ed3e8c2d4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:09.187718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:09.188428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:09.188723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:09.189008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:09.189726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:09.190042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:09.191409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:09.196561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 498/1000] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:09.202726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:09.308528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629340df-0d90-484f-8538-37fc7b5b5a76 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:09.325425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:09.344480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:09.345187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:09.345433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:09.345716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:09.346512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] RBAC: Adding query filter params (interface=public, service_id=089e6f7d02cf45448477d03b38aa1e63, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:09.346835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:09.348314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedcabf-d0fc-4fb8-9c07-eb30ed67b362 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:09.358453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 503/1001] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:43:09 2022] GET /identity/v3/endpoints?service_id=089e6f7d02cf45448477d03b38aa1e63&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:09.359597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:09.404470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:09.464626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a97369-c58a-43e4-8f1e-bdaa0b6126cc None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:09.504641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:09.505737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:09.505737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:09.506003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:09.506994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:09.508410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28a746-604c-44f5-8c40-d432b8be7bc0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:09.523920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 499/1002] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:43:09 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:43:10.221093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fe78-9010-4966-b9d0-ae7ef12ca399 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:10.221472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fe78-9010-4966-b9d0-ae7ef12ca399 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:10.221672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67fe78-9010-4966-b9d0-ae7ef12ca399 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:10.223180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 504/1003] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:10.228889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:10.229185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:10.229466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:10.247893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:10.249362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:10.317322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 500/1004] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:10.339327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67fe78-9010-4966-b9d0-ae7ef12ca399 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:10.438142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67fe78-9010-4966-b9d0-ae7ef12ca399 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:10.470444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:10.471280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:10.471513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:10.471789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] PATH_INFO: `/v3/services/image` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:10.472467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:10.473825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:10.485292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:10.486270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:10.487130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:10.491902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 505/1005] 10.222.0.24 () {56 vars in 1105 bytes} [Tue May 24 13:43:10 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:10.495442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:10.533081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:10.599171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0939854-3711-4556-a40d-26ca2be35f69 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:10.634017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:10.634699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:10.634991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:10.635269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:10.636032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:10.636347 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:10.637764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad7c45-69e7-4525-b279-369f4a03b7f7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:10.643636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 501/1006] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:10.649380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:10.698641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:10.757030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de2f6-046a-4a24-82f4-0bdb5d0ed604 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:10.791880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:10.792548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:10.792831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:10.793107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:10.793811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] RBAC: Adding query filter params (type=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:10.794117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:10.795488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c88bb-0b80-433c-904d-91fc446a3894 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:10.804401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 506/1007] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:11.284606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03003-e789-44a4-aead-443655988fba None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.285004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03003-e789-44a4-aead-443655988fba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.285184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f03003-e789-44a4-aead-443655988fba None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.286686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 502/1008] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:11.292174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.292467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.292749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.310922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:11.312129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:11.382894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 507/1009] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:11.405681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f03003-e789-44a4-aead-443655988fba None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:11.528342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:11.532645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f03003-e789-44a4-aead-443655988fba None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:11.570571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:11.571315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.571603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.571919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.578446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:11.580374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0c1e2-c946-4604-9d35-ff66cd37a6ef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:11.581492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 503/1010] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:11.587466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:11.628624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:11.689894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed96ce4-a5c2-4973-bf64-a4a962338e87 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:11.723895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:11.724574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.724927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.725208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.725932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:11.727371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e3fca1-d00d-478f-87bb-c7ebd949c73d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:11.753928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 508/1011] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:43:11 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:11.768426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddc58-1d16-4c85-88a0-684aab0c345c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.768861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddc58-1d16-4c85-88a0-684aab0c345c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.769256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ddc58-1d16-4c85-88a0-684aab0c345c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.771013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 504/1012] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:11.777487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:11.777863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:11.778164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:11.796065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:11.797301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:11.876576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 509/1013] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:11 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:11.900161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ddc58-1d16-4c85-88a0-684aab0c345c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:12.014149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ddc58-1d16-4c85-88a0-684aab0c345c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:12.049431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:12.050503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:12.050951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:12.051377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:12.052230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:12.054241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:12.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:12.067561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 13:43:12.068930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:12.071461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 505/1014] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:12.077192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:12.100283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:12.195658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f6006-1896-448d-959c-d655fd0e2c87 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:12.230296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:12.231244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:12.231732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:12.232238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:12.233121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:12.233610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:12.235148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:12.240779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 510/1015] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:12.246119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:12.279727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:12.353596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db9a7d-caeb-46b4-9c00-4a18de0e0530 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:12.388103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:12.388766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:12.389049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:12.389324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:12.390147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:12.390456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:12.391862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a64fff9-abe1-46c3-a71e-5ba0cd295347 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:12.401689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 506/1016] 10.222.0.24 () {56 vars in 1112 bytes} [Tue May 24 13:43:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:12.403174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:12.490191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:12.517172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a46c-2365-4af7-b072-2d59351f47ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:12.555086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:12.555752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:12.556073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:12.556413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:12.557162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:12.558535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de78389-18a6-4807-90fb-f0e496f9993f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:12.583851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 511/1017] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:43:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:13.065519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346a1-5642-4a50-833a-fc2f76f1ba4c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:13.066036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346a1-5642-4a50-833a-fc2f76f1ba4c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:13.066114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26346a1-5642-4a50-833a-fc2f76f1ba4c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:13.067496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 507/1018] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:13.073376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b156a6-e8e8-4293-8347-8db7ca1a5c30 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:13.073782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b156a6-e8e8-4293-8347-8db7ca1a5c30 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:13.073950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b156a6-e8e8-4293-8347-8db7ca1a5c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:13.090920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-56b156a6-e8e8-4293-8347-8db7ca1a5c30 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:13.092139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-56b156a6-e8e8-4293-8347-8db7ca1a5c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:13.165020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 512/1019] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:13 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:13.188099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26346a1-5642-4a50-833a-fc2f76f1ba4c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:13.288804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:13.296788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26346a1-5642-4a50-833a-fc2f76f1ba4c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:13.330793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:13.331385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:13.331678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:13.332004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:13.332746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:13.334634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451735e2-c284-4638-9e3c-a68183f9b0f6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:13.355853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 508/1020] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:43:13 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:14.056744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7f8b-b1f5-456a-a6f6-d2fe8099964d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.057127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7f8b-b1f5-456a-a6f6-d2fe8099964d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.057326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded7f8b-b1f5-456a-a6f6-d2fe8099964d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.058737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 513/1021] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:14.063869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.064171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.064453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.081273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:14.082470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:14.156397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 509/1022] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:14.181598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7f8b-b1f5-456a-a6f6-d2fe8099964d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:14.250816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:14.289380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded7f8b-b1f5-456a-a6f6-d2fe8099964d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:14.324465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:14.325186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.325475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.325756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.326935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:14.328546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b58737-e39f-4a46-a8f8-ca6d4a8a1474 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:14.330321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 514/1023] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:14.335974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:14.422010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:14.442860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8245ed9-7f3a-44f8-9d50-cd93f735cc36 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:14.477918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:14.478618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.478910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.479191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.479982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:14.481480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae374f-7cfe-4205-add4-2f6f7cfdaf42 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:14.514932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 510/1024] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:43:14 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:14.893417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae684c-7fd2-49c6-b4bd-7b574b26c585 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.893787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae684c-7fd2-49c6-b4bd-7b574b26c585 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.893973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae684c-7fd2-49c6-b4bd-7b574b26c585 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.895364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 515/1025] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:14.901085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:14.901387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:14.901674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:14.919027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:14.920262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:14.991856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 511/1026] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:15.013401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae684c-7fd2-49c6-b4bd-7b574b26c585 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.119784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae684c-7fd2-49c6-b4bd-7b574b26c585 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.158612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.159341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.159625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.159925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.161820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.172642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:15.181662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:15.182992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:15.184346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:15.185423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 516/1027] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:15.188756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.256744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.292178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9301896-6339-4c69-b637-1bb165e77bf8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.326571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.327239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.327523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.327798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.328574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:15.328881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:15.330236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:15.339550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 512/1028] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:15.344951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.451254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789db41f-5aab-49a0-a347-6d0eff7ca954 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.485729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.486468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.486747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.487023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.499539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:15.501028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3d633301-11e9-4bf2-802a-e56b96cfa80d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:15.520995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:15.522706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:15.523581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:15.523581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:15.523581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:15.523581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:15.523581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 517/1029] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:15.529492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.609725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b3f8d-438c-4509-acfa-d8e6585a97b3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.610049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b3f8d-438c-4509-acfa-d8e6585a97b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.610294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b3f8d-438c-4509-acfa-d8e6585a97b3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.611782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 518/1030] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:15.617117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.617407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.617684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.633329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed50b99-d3e4-4851-ad9d-7b3375445ff4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.634608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:15.635857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:15.664167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.670814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.671504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.671792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.672115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.672841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:15.673155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:15.674541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:15.684989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.693514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 513/1031] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:15.695054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.716942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 519/1032] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:15.739213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.806143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7f717c-3ee6-4d62-ae75-b9b9a5d1d690 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.843509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.844382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.844668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.844945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.848630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c6f80-d5f0-4233-bbce-19e017f327b4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:15.852689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:15.854126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:15.870072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:15.871196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:15.872204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 514/1033] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:15.876887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.886812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:15.887604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:15.887919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:15.888197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:15.888852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:15.890212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:15.901980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:15.902943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:15.903785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 520/1034] 10.222.0.24 () {56 vars in 1105 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:15.908903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:15.935297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.953597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:15.987360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a043507-8aba-4c06-a66e-1d00d13db591 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.017485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb7a94d-c3ad-47cd-bef6-bb2a097c38a0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.024140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.024807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.025093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.025373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.026213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:16.026525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.027980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.044470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 515/1035] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:16.050012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.060811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.061480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.061765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.062053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.062749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:16.063054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.064453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.075253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 521/1036] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:16.076531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.152212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7640ef-738b-4172-b3f1-300ecdae21f4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.170027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.194956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2d8fd-d06a-41f5-8dc9-2689ed8ec71a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.197973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.200037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.200712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.200986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.201409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.232134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.232804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.233091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.233368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.234085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] RBAC: Adding query filter params (type=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:16.234392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.235866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.246897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 522/1037] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:16.248021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.278184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.279633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.350476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d1768-ce03-4fba-bebb-3a7c6910b299 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.386945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.387615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 516/1038] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 13:43:16.388200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.388572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.388869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.389691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] RBAC: Adding query filter params (interface=public, service_id=be3e29b7d4b843a48da2395bc068c5f6, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:16.390013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.391395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.392981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.402797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 523/1039] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/endpoints?service_id=be3e29b7d4b843a48da2395bc068c5f6&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:16.403781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.404223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.436965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.518600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284870-0ac6-4c89-9906-ef7684ec3d6e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.531358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0712917-5890-4597-89fb-658f6c04daa6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.555346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.556171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.556468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.556745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.567581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.567976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.569251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.569532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.569766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.569877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.570476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.571845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.578447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 524/1040] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:16.587883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:16.588860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:16.589736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:16.589736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:16.589736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:16.589736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:16.592206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 517/1041] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:16.595080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.706451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.713949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880d2302-5306-48bd-94db-c7ed9ec25184 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.752526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.753207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.753495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.753771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.754565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:16.754877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.757500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f527d-16d8-4905-ba62-e7a8aaeb0a86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.771591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 525/1042] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:16.777394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.827663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9fac-a6ec-4ea3-b8b7-f363ed006764 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.827982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9fac-a6ec-4ea3-b8b7-f363ed006764 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.828265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a9fac-a6ec-4ea3-b8b7-f363ed006764 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.829621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 526/1043] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:16.834856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.835145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.835424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.839126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:16.852667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:16.853842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:16.886780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e35d4a-adb5-469a-861e-f4744bbded4a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:16.921169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 527/1044] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:16.923135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:16.923965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:16.924257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:16.924539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:16.931496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:16.932978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:16.944712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:16.947334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:16.948432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 518/1045] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:16.954190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.039111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.060063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2be34-73e4-4b94-867c-44933c45c4bc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.063106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3cd423-1b48-4ddd-b356-324b4abefae2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.084321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.099838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.100515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.100805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.101115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.101876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:17.102190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.102382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.103142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.103421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.103621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.103732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.114261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:17.117369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 519/1046] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:17.122658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:17.123574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:17.125038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:17.125695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.125775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 528/1047] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:17.129723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.231295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1717-6d0d-4eda-9803-e53aa1104824 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.242280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3e80a-5976-41da-ac34-3551f5d267e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.269036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.269668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.269950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.270378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.271046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] RBAC: Adding query filter params (role.id=ca62ce8eb7e94f379ae605ecd85a3683, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:17.271904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.273579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.278930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.279444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 520/1048] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=ca62ce8eb7e94f379ae605ecd85a3683 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:17.280232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.280603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.280964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.281760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:17.282083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.283494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.295962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 529/1049] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:17.299313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.351679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.411852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff3439-6c23-43ca-ae4a-a6bd2307e392 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.449683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.450570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.450857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.451131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] PATH_INFO: `/v3/users/0f7a30c9e177479bacdc43f8865f1fd0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.468348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] RBAC: Authorizing `identity:get_user(user_id=0f7a30c9e177479bacdc43f8865f1fd0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.470103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.471766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 521/1050] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/users/0f7a30c9e177479bacdc43f8865f1fd0 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:17.477215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.489083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.586838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d0cc60-0fb6-4baa-901b-519126aae4aa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.619001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.619654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.619996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.620268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.686682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.688153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.698598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.803896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 530/1051] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:17.805771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:17.907250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e384b6-eb18-4d13-99e1-ab04f6d7eccd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:17.908716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:17.942031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:17.942884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:17.943713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:17.944032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:17.945171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:17.946598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb5eb0-8d04-4012-a4ac-e895e0e21b61 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:17.953145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 522/1052] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:17.954204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:18.052930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d344a0-dc95-416e-8f36-602281c60a13 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:18.062560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:18.088976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:18.089910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:18.090396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:18.090883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:18.092458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=ce42842904c44017b388f2860088ff29, user.id=0f7a30c9e177479bacdc43f8865f1fd0) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:18.092947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:18.094633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a9a60-8581-46a5-b9c0-d3e408634356 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:18.108197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 531/1053] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:17 2022] GET /identity/v3/role_assignments?user.id=0f7a30c9e177479bacdc43f8865f1fd0&scope.project.id=ce42842904c44017b388f2860088ff29&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:18.842895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6a81-3da8-4040-998f-71dd856c8685 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:18.843391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6a81-3da8-4040-998f-71dd856c8685 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:18.843391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa6a81-3da8-4040-998f-71dd856c8685 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:18.844940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 523/1054] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:18.850406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:18.850796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:18.850981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:18.868019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:18.869207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:18.939371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 532/1055] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:18.963882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa6a81-3da8-4040-998f-71dd856c8685 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.067285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa6a81-3da8-4040-998f-71dd856c8685 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.101994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.102721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.103012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.103293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.103982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:19.105352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:19.117019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:19.117995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 13:43:19.118824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:19.123181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 524/1056] 10.222.0.24 () {56 vars in 1105 bytes} [Tue May 24 13:43:18 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:19.124476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.225187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3af1ab-092e-4348-aabb-4efb7540df15 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.226781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:19.265113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.265854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.266139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.266438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.267143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:19.267593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:19.269036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:19.278479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 533/1057] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:19.280047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.318258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:19.389370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e61e1-c710-4b28-a59b-6ade6122efb6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.423412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f54857-7a04-4705-8554-4827e343240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.424106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54857-7a04-4705-8554-4827e343240d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.424399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54857-7a04-4705-8554-4827e343240d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.424683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f54857-7a04-4705-8554-4827e343240d None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.425395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f54857-7a04-4705-8554-4827e343240d None admin] RBAC: Adding query filter params (type=image) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:19.425709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f54857-7a04-4705-8554-4827e343240d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:19.427082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f54857-7a04-4705-8554-4827e343240d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:19.432646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 525/1058] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:19.438220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.507063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea67-d127-46ae-8b74-961e784189e9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.508285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea67-d127-46ae-8b74-961e784189e9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.508285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0ea67-d127-46ae-8b74-961e784189e9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.510298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 526/1059] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:43:19.514752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.515044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.515326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.532123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:19.533317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:19.542896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079c0b-86c2-4792-bef8-d22a026ed788 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.577221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.577905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.578194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.578480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.579229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:19.580632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:19.599250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 534/1060] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:43:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:19.613704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 527/1061] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:19.636890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.666252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:19.753315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088074da-5f91-432a-bb9f-b8b460855c41 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.791091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.791949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.792244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.792525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.803183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:19.811568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:19.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:19.813706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 535/1062] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:43:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:19.819568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:19.846056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:19.926506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f4c8-3fb4-4b3b-aaec-9e131f8d3af7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:19.962338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:19.963021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:19.963311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:19.963593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:19.964389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:19.964787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:19.966230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:19.980116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 528/1063] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:19.981723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.065948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.086102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fde59-3744-4f47-9e88-66d0a530fb89 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.124012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.125173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.125703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.126191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.138505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.140364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:20.159018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:20.160037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:20.160943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:20.160943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:20.160943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:20.160943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:20.168599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 536/1064] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:20.170774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.269311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.287939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56248e-5923-44b6-be5a-ca6af97fcff2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.324569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.325593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.326093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.326577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.327541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:20.328098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.329678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b34ffb-a051-4093-b421-b6a4f95f896a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.347940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 529/1065] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:20.349971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.361497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d755e0d-c2d7-4d29-9547-39801325de86 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.361835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d755e0d-c2d7-4d29-9547-39801325de86 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.362065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d755e0d-c2d7-4d29-9547-39801325de86 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.363692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 530/1066] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:20.369827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.370119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.370399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.387332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:20.388554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:20.467880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 531/1067] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:20.483848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42f6bc-4e5b-4fe7-a2ff-ec28a6c26192 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.486635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.531063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.532757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.533256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.533762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.536733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.551158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.552456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:20.568598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:20.569829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:20.570757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 537/1068] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:20.575321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.593184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.619786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc46d97-dddb-4a17-b511-4136e328b07c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.655160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.655987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.656279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.656557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.667563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:20.675495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:20.676444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:20.677333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 532/1069] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:20.682625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.687166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9b8a8-6c26-4abe-ada6-703890732f92 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.724593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.725269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.725790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.726082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.727053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:20.727375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.728971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.753809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 538/1070] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:20.755377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.793028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ddd63-7ce9-4690-a809-1d6d1a7d960f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.810800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.835547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.836263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.836553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.836835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.838060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:20.838377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.840465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.844176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:20.858464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 533/1071] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:20.860203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.860793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a253321-cd88-4e02-b1f7-1dea983f29fa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:20.899493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:20.900344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:20.901356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:20.901356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:20.911792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:20.913288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:20.931435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:20.932372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:20.933246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:20.933246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:20.933246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:20.933246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:20.935569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 539/1072] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:20.938228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:20.975163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8108527-37bf-4d57-9ba8-a63a4d2fe038 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.012668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.013546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.013832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.014113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] PATH_INFO: `/v3/users/0f7a30c9e177479bacdc43f8865f1fd0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.015382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] RBAC: Authorizing `identity:get_user(user_id=0f7a30c9e177479bacdc43f8865f1fd0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.016851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.023688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 534/1073] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/users/0f7a30c9e177479bacdc43f8865f1fd0 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:21.024766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:21.052376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0122d49-54cd-4a87-ae13-85ae2bb522ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.055651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:21.092499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:21.093077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.094659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.095185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.095635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.096523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:21.096975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.098479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.111400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 540/1074] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:21.116962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:21.133615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509256c-b969-4c81-883e-537d2b93e86f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.171271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.172178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.172524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.172813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.174210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.175644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.177830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 535/1075] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:21 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:21.183453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:21.221607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a446eb4-ef7c-481e-925f-579119530ed6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.258571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628078cd-e730-4047-a164-662c62402452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.259340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628078cd-e730-4047-a164-662c62402452 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.259622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628078cd-e730-4047-a164-662c62402452 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.259925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628078cd-e730-4047-a164-662c62402452 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.267016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628078cd-e730-4047-a164-662c62402452 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.268526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628078cd-e730-4047-a164-662c62402452 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-628078cd-e730-4047-a164-662c62402452 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:21.283920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:21.284849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:21.285718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 541/1076] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:21.291596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628078cd-e730-4047-a164-662c62402452 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:21.295703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132fe6-5030-4849-8ac1-b7ed7fea1ea9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.323272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:21.330750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.331641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.331957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.332233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29/users/0f7a30c9e177479bacdc43f8865f1fd0/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.334624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce42842904c44017b388f2860088ff29, user_id=0f7a30c9e177479bacdc43f8865f1fd0, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.336107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.338336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:21.352691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 536/1077] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:43:21 2022] PUT /identity/v3/projects/ce42842904c44017b388f2860088ff29/users/0f7a30c9e177479bacdc43f8865f1fd0/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:43:21.414543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628078cd-e730-4047-a164-662c62402452 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.453026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.453713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.454000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.454281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.455092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:21.455403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.456942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d03d23-7d2b-47e8-924d-de0cd21c8473 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.475160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 542/1078] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:21.476576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:21.596583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755062a3-e1d0-449b-8b60-db3f803b0054 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:21.632726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:21.633619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.633902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.634179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/ca62ce8eb7e94f379ae605ecd85a3683` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.636445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=5bb056b4578442038d33d4fdf9de9584, role_id=ca62ce8eb7e94f379ae605ecd85a3683)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:21.637890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc718812-d924-4fe7-93b5-39e8fdd7e55d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:21.653382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 537/1079] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:43:21 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/5bb056b4578442038d33d4fdf9de9584/roles/ca62ce8eb7e94f379ae605ecd85a3683 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:43:21.894429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42d5d-a828-47f4-976f-8518e9587885 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.894672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42d5d-a828-47f4-976f-8518e9587885 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.895020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e42d5d-a828-47f4-976f-8518e9587885 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.897044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 543/1080] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:21.902506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:21.902940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:21.903104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:21.920317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:21.921514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:21.972130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:21.992540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 538/1081] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:22.014937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e42d5d-a828-47f4-976f-8518e9587885 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:22.104216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:22.123108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e42d5d-a828-47f4-976f-8518e9587885 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:22.159691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:22.160298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:22.160584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:22.160858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:22.232914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:22.234428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:22.320804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:22.339213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 544/1082] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:22.347056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:22.449480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e165c9-53f8-497a-9bae-2b01f4935d03 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:22.485751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:22.486556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:22.486842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:22.487120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:22.499240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:22.500838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-a62a89a5-bcbb-4a6c-9490-f74815b19fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:22.524101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:22.525112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:22.525971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:22.525971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:22.525971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:22.525971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:22.527897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 539/1083] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:22.531534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:22.636484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee2b31-6230-45d0-b152-339501832432 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:22.672172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:22.672834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:22.673117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:22.673393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:22.674111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:22.674420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:22.675779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad289a1e-0d5f-4f16-a55e-6244451dee28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:22.695850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 545/1084] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:23.561496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a24a-621f-4e6f-a8e0-78fc5ea3d6d1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.562007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a24a-621f-4e6f-a8e0-78fc5ea3d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.562125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3a24a-621f-4e6f-a8e0-78fc5ea3d6d1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.563442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 540/1085] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:23.569199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.569584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.569760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.586664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:23.587849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:23.625969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:23.652723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 546/1086] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:23.675742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a24a-621f-4e6f-a8e0-78fc5ea3d6d1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:23.715425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:23.780168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3a24a-621f-4e6f-a8e0-78fc5ea3d6d1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:23.813157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:23.813950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.814239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.814521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.825125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0e098302-c8db-431f-b009-5d4b20d27acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:23.832692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:23.833731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:23.834576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 541/1087] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:23.840016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:23.876922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf1edf-7325-4c84-a762-6cd49d257508 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.877214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf1edf-7325-4c84-a762-6cd49d257508 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.877499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf1edf-7325-4c84-a762-6cd49d257508 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.878868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 542/1088] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:43:23.884907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.885197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.885479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.902785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:23.904004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:23.938704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de9489-9ee9-4813-b183-431692b761a0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:23.970800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 543/1089] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:23.972041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:23.972708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:23.972996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:23.973273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:23.974067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:23.974376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:23.975854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:23.983838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 547/1090] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:23.989459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:23.996151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.016948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.032137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.091163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec65f57-0242-4ecb-b440-3d99e4159880 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.096110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598d399c-9cf1-4127-9414-ed1ba15782fe None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.126861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.127727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.128042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.128319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] PATH_INFO: `/v3/users/0f7a30c9e177479bacdc43f8865f1fd0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.129106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.129495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] RBAC: Authorizing `identity:get_user(user_id=0f7a30c9e177479bacdc43f8865f1fd0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.129916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.130792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.131356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.131768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.133427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 544/1091] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:23 2022] GET /identity/v3/users/0f7a30c9e177479bacdc43f8865f1fd0 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:24.139070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.144347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:24.152831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:24.153774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:24.154663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 548/1092] 10.222.0.24 () {56 vars in 1120 bytes} [Tue May 24 13:43:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:24.159745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.237195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.245473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80215c-307a-4627-808f-2eb28352cf1e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.254078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.261324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc6fe9-6e4e-4769-bf18-fa55a55da1b6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.279586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.280266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.280559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.280839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.295357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.295986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.296276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.296554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.297303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:24.297613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.298990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.307023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 549/1093] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:24.312417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.348330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.349815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.409648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ff2d7-37dd-45cc-987d-b53037975292 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.440113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.444213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.445011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.445298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.445575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.448478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 545/1094] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 13:43:24.454260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.456343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.457790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:24.475594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:24.476833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:24.477762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:24.477762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:24.477762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:24.477762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:24.477762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 550/1095] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:24.482374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.496097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.557449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8ca8-0702-4fbd-8aa5-4e21cd5f0307 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.587361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239ca32b-0c83-49a4-a864-7f126ea4c8de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.591624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.592516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.592797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.593069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.594236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.595656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.597402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 546/1096] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:24.603837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.621587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.622255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.622538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.622816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.623534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:24.623871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.625236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.637217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 551/1097] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:24.642579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.664787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.702138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363323b3-dc96-47a4-8bba-bfa610b36ca2 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.711784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.736837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.737482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.737768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.738047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.738873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=ce42842904c44017b388f2860088ff29, user.id=0f7a30c9e177479bacdc43f8865f1fd0) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:24.739188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.741500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.742462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b2a7d-705e-4b17-a109-0efcc467f217 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.747331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 547/1098] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/role_assignments?user.id=0f7a30c9e177479bacdc43f8865f1fd0&scope.project.id=ce42842904c44017b388f2860088ff29&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:24.778698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.779528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.779845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.780123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.789400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.790900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:43:24.806705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:24.807805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:24.808770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 552/1099] 10.222.0.24 () {56 vars in 1093 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:24.814176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:24.918770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c022044-f9c0-48d4-b334-7afc07cb9670 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:24.945543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:24.954916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:24.955584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:24.955900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:24.956181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:24.957023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:24.957327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:24.958972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:24.976696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 548/1100] 10.222.0.24 () {56 vars in 1133 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:24.982512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:25.083081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c38fee-45ce-4360-a762-014a7268c002 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:25.111367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:25.118655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:25.119304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:25.119590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:25.119898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:25.185356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:25.186800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:25.285057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 553/1101] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:25.290737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:25.387702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faa65b6-c973-44df-888b-9c53b303b8b7 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:25.420416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:25.421208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:25.421497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:25.421778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:25.432308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:25.433754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:25.450940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:25.451949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:25.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:43:25.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:25.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:43:25.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:25.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 549/1102] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:25.457822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:25.488014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:25.558335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d343a5-1a94-4acd-a4e7-cbc6c242ae20 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:25.591805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:25.592491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:25.592776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:25.593052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:25.593770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:25.594079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:25.595458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:25.610942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 554/1103] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:25.612179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:25.647342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:25.710635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664b3f1-b82f-4ec6-a6df-73de06cc73a9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:25.742706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:25.743486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:25.743776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:25.744086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:25.750627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:25.752083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:43:25.766312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:25.767203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:25.768604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 550/1104] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:25.773331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:25.861031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:25.871805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0987-1ad6-42de-9caf-8106780aedac None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:25.907598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:25.908218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:25.908506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:25.908785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:25.909568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:25.909878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:25.911841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d43df7-14ba-427c-bfb4-94b6c4a5bcfc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:25.924377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 555/1105] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:25.930336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:26.029457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:26.033548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4263680-1f09-4b10-b76e-18402fc0efa3 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:26.067156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:26.068093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:26.068515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:26.068796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:26.069630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] RBAC: Adding query filter params (role.id=ca62ce8eb7e94f379ae605ecd85a3683, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=5bb056b4578442038d33d4fdf9de9584) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:26.069948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:26.071378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de786672-738b-4b75-89f2-28406bd10ac6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:26.077098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 551/1106] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:25 2022] GET /identity/v3/role_assignments?user.id=5bb056b4578442038d33d4fdf9de9584&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=ca62ce8eb7e94f379ae605ecd85a3683 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:27.044232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b6d16-d3ab-4fa8-bc33-493bfc4b952f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:27.044805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b6d16-d3ab-4fa8-bc33-493bfc4b952f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:27.044805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b6d16-d3ab-4fa8-bc33-493bfc4b952f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:27.046162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 556/1107] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:27.054631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:27.055045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:27.055202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:27.073395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:27.074595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:27.152388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 552/1108] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:27.183132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b6d16-d3ab-4fa8-bc33-493bfc4b952f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:27.297443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b6d16-d3ab-4fa8-bc33-493bfc4b952f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:27.335408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:27.336222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:27.336508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:27.336782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:27.337951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:27.339377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf788d16-2833-4001-b9f4-fcf962ef661d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:27.341011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 557/1109] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:27.346838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:27.447127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:27.457777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a3cfe-be8c-4cd1-a783-f440f2db0e21 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:27.496822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:27.497502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:27.497789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:27.498068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:27.498790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:27.500297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c70e000-bed1-4b47-ac12-a3fcb22051a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:27.524194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 553/1110] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:43:27 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:29.750070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a410-1785-4351-b3e8-03facb9d935a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:29.750616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a410-1785-4351-b3e8-03facb9d935a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:29.750616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1a410-1785-4351-b3e8-03facb9d935a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:29.752006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 558/1111] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:29.757620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:29.758016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:29.758112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:29.775073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:29.776293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:29.840622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 554/1112] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:29.863589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1a410-1785-4351-b3e8-03facb9d935a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:29.883410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:29.962509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1a410-1785-4351-b3e8-03facb9d935a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:29.995006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:29.995870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:29.996159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:29.996435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] PATH_INFO: `/v3/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:29.997886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] RBAC: Authorizing `identity:get_role(role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:29.999259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:30.000618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 559/1113] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:29 2022] GET /identity/v3/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:30.007199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:30.079468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:30.101808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6b5f8-a8b6-4b73-a353-7d51d584cc53 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:30.134805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:30.135664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:30.135979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:30.136260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] PATH_INFO: `/v3/users/2532fcd88d6a4b609002128c4c5c2eac` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:30.148729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=2532fcd88d6a4b609002128c4c5c2eac)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:30.150156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:30.151769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 555/1114] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:30 2022] GET /identity/v3/users/2532fcd88d6a4b609002128c4c5c2eac => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:30.157399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:30.240629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:30.257602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8d457-a437-4fc0-a504-9dbfaa1ef3c6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:30.294233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:30.294906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:30.295191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:30.295469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:30.360833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:30.362208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:30.443223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:30.458920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 560/1115] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:30.464828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:30.563291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24376203-6971-4639-b002-217e9088c4b9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:30.596703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:30.597562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:30.597848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:30.598127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:30.600137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:30.601568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f37c8-8999-46fc-aa4c-3f54cd5cb779 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:30.603278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 556/1116] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:30 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:30.609298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:30.704563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695ad2b-a704-4407-8467-5d0c1fbf9b57 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:30.738046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:30.738689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:30.738981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:30.739293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:30.740247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=ce42842904c44017b388f2860088ff29, user.id=2532fcd88d6a4b609002128c4c5c2eac) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:30.740567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:30.741995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f8ac1-b800-4b6b-96f0-a5d0d28bf40b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:30.747296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 561/1117] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:30 2022] GET /identity/v3/role_assignments?user.id=2532fcd88d6a4b609002128c4c5c2eac&scope.project.id=ce42842904c44017b388f2860088ff29&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:32.900158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced0a56-54c2-4a9f-96c8-c0361b9704a4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:32.900661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced0a56-54c2-4a9f-96c8-c0361b9704a4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:32.900661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced0a56-54c2-4a9f-96c8-c0361b9704a4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:32.902002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 557/1118] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:32.907121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:32.907533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:32.907678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:32.924835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:32.925994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:32.969812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:32.990209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 562/1119] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:33.012661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced0a56-54c2-4a9f-96c8-c0361b9704a4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:33.015514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:33.113929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced0a56-54c2-4a9f-96c8-c0361b9704a4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:33.147535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:33.148407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:33.148698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:33.148978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] PATH_INFO: `/v3/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:33.150492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] RBAC: Authorizing `identity:get_role(role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:33.151835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:33.153205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 558/1120] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:33 2022] GET /identity/v3/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:33.158896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:33.255459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bc3980-0e79-499b-8187-46f4aebd5783 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:33.288246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:33.289107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:33.289393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:33.289669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] PATH_INFO: `/v3/users/2532fcd88d6a4b609002128c4c5c2eac` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:33.290844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] RBAC: Authorizing `identity:get_user(user_id=2532fcd88d6a4b609002128c4c5c2eac)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:33.292312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:33.293926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 563/1121] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:33 2022] GET /identity/v3/users/2532fcd88d6a4b609002128c4c5c2eac => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:33.299797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:33.367301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:33.398323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523adf0a-8b85-4ea9-8bfc-ee99c37be2cb None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:33.430578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:33.431369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:33.431659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:33.431969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:33.433152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:33.434569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49fd1d-c4b1-4bbf-8da9-bef075e90827 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:33.436267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 559/1122] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:33 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:33.442775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:33.491237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:33.542736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944c815-7215-416d-a76e-875de877c2cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:33.579341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:33.580256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:33.580543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:33.580821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29/users/2532fcd88d6a4b609002128c4c5c2eac/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:33.583146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] RBAC: Authorizing `identity:create_grant(project_id=ce42842904c44017b388f2860088ff29, user_id=2532fcd88d6a4b609002128c4c5c2eac, role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:33.584615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39d480b-7c35-43b0-befc-a975b9240d3b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:33.598700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 564/1123] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:43:33 2022] PUT /identity/v3/projects/ce42842904c44017b388f2860088ff29/users/2532fcd88d6a4b609002128c4c5c2eac/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:43:35.769648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf9f26-7830-4738-86e5-08fcc24ee450 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:35.770681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf9f26-7830-4738-86e5-08fcc24ee450 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:35.770681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf9f26-7830-4738-86e5-08fcc24ee450 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:35.772004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 560/1124] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:35.777227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:35.777598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:35.777785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:35.794722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:35.795937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:35.860401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 565/1125] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:35.883665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf9f26-7830-4738-86e5-08fcc24ee450 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:35.982340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf9f26-7830-4738-86e5-08fcc24ee450 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:36.014683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:36.015521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:36.015804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:36.016110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] PATH_INFO: `/v3/roles/af2f185f6c6e4873bd58c0208ade25ff` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:36.017546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] RBAC: Authorizing `identity:get_role(role_id=af2f185f6c6e4873bd58c0208ade25ff)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:36.018911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:36.021125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 561/1126] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:35 2022] GET /identity/v3/roles/af2f185f6c6e4873bd58c0208ade25ff => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:36.026718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:36.064787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:36.122891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404e9ea-7dc8-4834-b672-b63fb0f36f3f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:36.156355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:36.157214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:36.157502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:36.157789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] PATH_INFO: `/v3/users/2532fcd88d6a4b609002128c4c5c2eac` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:36.158966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] RBAC: Authorizing `identity:get_user(user_id=2532fcd88d6a4b609002128c4c5c2eac)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:36.160427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:36.162175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 566/1127] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:36 2022] GET /identity/v3/users/2532fcd88d6a4b609002128c4c5c2eac => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:36.167791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:36.176304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:36.268116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f49eb-7bad-4feb-9ee3-f300126ee29e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:36.300996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:36.301900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:36.302261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:36.302544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:36.367255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:36.368718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:36.375862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:36.464549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 562/1128] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:36.470371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:36.569520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947994fd-562b-44e8-8767-f84e108070ba None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:36.576479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:36.602493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:36.603705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:36.604197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:36.604649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] PATH_INFO: `/v3/projects/ce42842904c44017b388f2860088ff29` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:36.606512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] RBAC: Authorizing `identity:get_project(project_id=ce42842904c44017b388f2860088ff29)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:36.609127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83397c1-4713-47a2-9da0-6bd79a3dafd1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:36.611051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 567/1129] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:36 2022] GET /identity/v3/projects/ce42842904c44017b388f2860088ff29 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:36.617869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:36.714646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f256d-573b-48a5-ac3c-a1332563282c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:36.721593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:36.748234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:36.748880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:36.749165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:36.749444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:36.750266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] RBAC: Adding query filter params (role.id=af2f185f6c6e4873bd58c0208ade25ff, scope.project.id=ce42842904c44017b388f2860088ff29, user.id=2532fcd88d6a4b609002128c4c5c2eac) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:36.750581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:36.752037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe0d0a-ccb1-4544-95a1-d40511fc3b15 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:36.758214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 563/1130] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:36 2022] GET /identity/v3/role_assignments?user.id=2532fcd88d6a4b609002128c4c5c2eac&scope.project.id=ce42842904c44017b388f2860088ff29&role.id=af2f185f6c6e4873bd58c0208ade25ff => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:38.960685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291129f-e7cd-45e8-9e63-5fd032095195 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:38.961198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291129f-e7cd-45e8-9e63-5fd032095195 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:38.961198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291129f-e7cd-45e8-9e63-5fd032095195 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:38.962531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 568/1131] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:38.967749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:38.968170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:38.968346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:38.985013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:38.986186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:39.052379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 564/1132] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:39.075579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291129f-e7cd-45e8-9e63-5fd032095195 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:39.175545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291129f-e7cd-45e8-9e63-5fd032095195 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:39.208419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:39.209190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:39.209474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:39.209743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:39.210933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:39.212327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da6f89b-0284-40e2-8d46-759e904bfd95 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:39.213889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 569/1133] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:39.219648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:39.313541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:39.316490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604c081-7343-4722-8331-b8d8c7eb0b36 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:39.350282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:39.350939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:39.351223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:39.351500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:39.352246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:39.353665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87dfe8-a3f0-4897-9593-a6b428e08c58 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:39.378279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 565/1134] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:43:39 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:41.588898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df35f-62e1-47b9-9281-8e5b8bccae5c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:41.589414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df35f-62e1-47b9-9281-8e5b8bccae5c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:41.589518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4df35f-62e1-47b9-9281-8e5b8bccae5c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:41.590837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 570/1135] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:41.596094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:41.596456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:41.596669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:41.613337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:41.614612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:41.680087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 566/1136] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:41.704884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4df35f-62e1-47b9-9281-8e5b8bccae5c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:41.749943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:41.806862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4df35f-62e1-47b9-9281-8e5b8bccae5c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:41.839524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:41.840331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:41.840615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:41.840891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:41.842058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:41.843493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3432d88-1357-42c1-9094-1cd7e4b980ba None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:41.845152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 571/1137] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:41.851480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:41.943454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:41.966217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c07a10-63d2-452f-af6b-10cadc7e2258 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:42.001615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:42.002271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:42.002554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:42.002829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:42.003544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:42.005376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e7f01-b1d2-44a0-b654-d886566178f1 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:42.024777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 567/1138] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:43:41 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:44.247484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9dd-a4e9-4e17-bee4-8375684db37f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.248072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9dd-a4e9-4e17-bee4-8375684db37f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.248072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262e9dd-a4e9-4e17-bee4-8375684db37f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.249422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 572/1139] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:44.255195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.255910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.256089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.272500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:44.273680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:44.338193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 568/1140] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:44.360851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e9dd-a4e9-4e17-bee4-8375684db37f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:44.459009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262e9dd-a4e9-4e17-bee4-8375684db37f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:44.469135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:44.496275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:44.497062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.497348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.497675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.508083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:44.515640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:44.516898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:44.518146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 573/1141] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:43:44.522822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:44.589518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:44.623150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc13bd-d6c2-4d78-9376-84ca4fa89164 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:44.655647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:44.656341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.656633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.656913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.657666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:44.657979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:44.659349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:44.667811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 569/1142] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:43:44.673671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:44.770802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5f18f-c8a8-4297-b44a-073b7e1fa7ff None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:44.803203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:44.804097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.804386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.804666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] PATH_INFO: `/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:44.817397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] RBAC: Authorizing `identity:get_user(user_id=3064c7fc4f714eccb6db9c70ad3bbc57)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:44.818832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:44.820483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 574/1143] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:44 2022] GET /identity/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:44.826174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:44.924367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeeebe2-4b40-4aaf-9885-bfc3699b3e69 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:44.953478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:44.960677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:44.961328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:44.961619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:44.961901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:45.027012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:45.028422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:45.123227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 570/1144] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:45.129862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:45.157786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:45.228446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f424ad-5ff1-4032-9547-3268297efb53 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:45.261434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:45.262288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:45.262577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:45.262859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] PATH_INFO: `/v3/projects/3da1d01c65a64964b158560d0d7eab6d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:45.264073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] RBAC: Authorizing `identity:get_project(project_id=3da1d01c65a64964b158560d0d7eab6d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:45.265508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26d83b-f82c-438a-9f9b-c01777bfb1bb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:45.267181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 575/1145] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:45 2022] GET /identity/v3/projects/3da1d01c65a64964b158560d0d7eab6d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:45.273838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:45.302330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:45.368366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab477c-f0df-49e8-b261-b7f73921c04d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:45.401511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:45.402146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:45.402431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:45.402713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:45.403532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=3da1d01c65a64964b158560d0d7eab6d, user.id=3064c7fc4f714eccb6db9c70ad3bbc57) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:45.403877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:45.405432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9e5f-7c75-4a0a-a15b-130b1aa68163 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:45.410854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 571/1146] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:45 2022] GET /identity/v3/role_assignments?user.id=3064c7fc4f714eccb6db9c70ad3bbc57&scope.project.id=3da1d01c65a64964b158560d0d7eab6d&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:47.580190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516309af-76c2-427b-baad-711b32e6997e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:47.580695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516309af-76c2-427b-baad-711b32e6997e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:47.580811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516309af-76c2-427b-baad-711b32e6997e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:47.582715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 576/1147] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:47.587969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:47.588362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:47.588548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:47.604884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:47.606064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:47.670996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 572/1148] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:47.694302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516309af-76c2-427b-baad-711b32e6997e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:47.785810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:47.792958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516309af-76c2-427b-baad-711b32e6997e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:47.827023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:47.827802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:47.828118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:47.828401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:47.838505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:47.846070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:47.847059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:47.848290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 577/1149] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:47.853445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:47.877715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:47.950206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358efcd-1a15-4584-8442-ef8268021f02 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:47.983192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:47.983875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:47.984168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:47.984452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:47.985208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:47.985523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:47.986890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:47.995211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 573/1150] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:48.001010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:48.098351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75033-f8e9-43a2-b8f9-2d61f3eb3bf6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:48.131631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:48.132508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:48.132794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:48.133128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] PATH_INFO: `/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:48.134311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] RBAC: Authorizing `identity:get_user(user_id=3064c7fc4f714eccb6db9c70ad3bbc57)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:48.135741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:48.137354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 578/1151] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:48 2022] GET /identity/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:48.143005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:48.225615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:48.241348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb864f79-3bea-4f5a-9368-804153b71dd9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:48.273882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:48.274734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:48.275021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:48.275301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] PATH_INFO: `/v3/projects/3da1d01c65a64964b158560d0d7eab6d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:48.276484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] RBAC: Authorizing `identity:get_project(project_id=3da1d01c65a64964b158560d0d7eab6d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:48.277914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8987c-4849-4f20-b930-f2e11a1c24ea None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:48.279596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 574/1152] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:48 2022] GET /identity/v3/projects/3da1d01c65a64964b158560d0d7eab6d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:48.285887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:48.311105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:48.383353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd422497-8e23-460a-b9e9-535ac70370e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:48.415641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:48.416540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:48.416826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:48.417104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] PATH_INFO: `/v3/projects/3da1d01c65a64964b158560d0d7eab6d/users/3064c7fc4f714eccb6db9c70ad3bbc57/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:48.419023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3da1d01c65a64964b158560d0d7eab6d, user_id=3064c7fc4f714eccb6db9c70ad3bbc57, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:48.420473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c04d54-5d30-44c7-b885-df5b3451cda9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:48.433510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 579/1153] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:43:48 2022] PUT /identity/v3/projects/3da1d01c65a64964b158560d0d7eab6d/users/3064c7fc4f714eccb6db9c70ad3bbc57/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:43:50.617934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493eb8c-ecde-4b48-b0aa-2e02733463b0 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:50.618446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493eb8c-ecde-4b48-b0aa-2e02733463b0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:50.618535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493eb8c-ecde-4b48-b0aa-2e02733463b0 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:50.619897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 575/1154] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:50.625009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:50.625439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:50.625583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:50.643009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:50.645000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:50.709042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 580/1155] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:50.732496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493eb8c-ecde-4b48-b0aa-2e02733463b0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:50.832310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493eb8c-ecde-4b48-b0aa-2e02733463b0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:50.864864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:50.865643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:50.865930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:50.866208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:50.876433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:43:50.884540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:50.885458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:43:50.886275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 576/1156] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:50.891667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:50.906841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:50.989678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30519f7-96c1-4c8b-bf07-d8ead037fe5c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:51.022604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:51.023265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:51.023557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:51.023868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:51.024626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:51.024941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:51.026314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:51.034186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 581/1157] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:51.040287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:51.055806 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:51.135898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d0dd2-45b6-430a-8cfc-a69bb31e2a90 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:51.168187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:51.169042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:51.169320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:51.169590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] PATH_INFO: `/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:51.170744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] RBAC: Authorizing `identity:get_user(user_id=3064c7fc4f714eccb6db9c70ad3bbc57)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:51.172253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:51.173840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 577/1158] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:43:51 2022] GET /identity/v3/users/3064c7fc4f714eccb6db9c70ad3bbc57 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:51.179561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:51.255974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:51.278917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be11bfe-9e65-40da-bbcd-0742ae7fe4e6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:51.310813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:51.311451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:51.311737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:51.312043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:51.375889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:51.377316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:51.449355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:51.472259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 582/1159] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:51.478164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:51.557647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:51.576965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c7882-16bf-4a57-9271-32bb0a82b20f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:51.611231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:51.612104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:51.612387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:51.613345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] PATH_INFO: `/v3/projects/3da1d01c65a64964b158560d0d7eab6d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:51.614582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] RBAC: Authorizing `identity:get_project(project_id=3da1d01c65a64964b158560d0d7eab6d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:51.616036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475592-586b-46ad-8557-2fe0371ac784 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:51.617718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 578/1160] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:43:51 2022] GET /identity/v3/projects/3da1d01c65a64964b158560d0d7eab6d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:51.623750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:51.720395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c972c7-bc14-4a78-822e-230f34710ada None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:51.753067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:51.754279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:51.754562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:51.754838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:51.755964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=3da1d01c65a64964b158560d0d7eab6d, user.id=3064c7fc4f714eccb6db9c70ad3bbc57) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:51.756281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:51.757712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a41b6-cdd4-43b2-bcc7-a668564340d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:51.763382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 583/1161] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:43:51 2022] GET /identity/v3/role_assignments?user.id=3064c7fc4f714eccb6db9c70ad3bbc57&scope.project.id=3da1d01c65a64964b158560d0d7eab6d&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:53.964511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda678-3479-401e-b7ac-f5db4302b622 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:53.965010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda678-3479-401e-b7ac-f5db4302b622 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:53.965010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dda678-3479-401e-b7ac-f5db4302b622 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:53.966356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 579/1162] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:53.971568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:53.972025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:53.972310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:53.989176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:53.990350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:54.023775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:54.056250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 584/1163] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:54.078554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dda678-3479-401e-b7ac-f5db4302b622 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:54.177029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dda678-3479-401e-b7ac-f5db4302b622 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:54.208980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:54.209612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:54.209899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:54.210178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:54.228362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:54.275596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:54.277051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:54.371465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 580/1164] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:43:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:43:54.377411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:54.473409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecf4c0-bc0c-4711-949f-21bf999d6be3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:54.506715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:54.507335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:54.507618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:54.507925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:54.514516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:54.515981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-47e9232f-ac84-467c-aa73-3bfcc287dfe4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:43:54.530374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:43:54.531560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:43:54.532639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 585/1165] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:43:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:43:54.538288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:54.586814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:54.634776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ff64a8-724a-4ea9-9b75-a04fd1f24955 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:54.667469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:54.668147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:54.668434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:54.668713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:54.669435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:43:54.669752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:54.671101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d0e61-2270-48a4-afae-57eb47e15fef None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:54.678640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 581/1166] 10.222.0.24 () {56 vars in 1115 bytes} [Tue May 24 13:43:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:56.835478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6a32-8740-4a35-b0c5-860497be272c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:56.836108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6a32-8740-4a35-b0c5-860497be272c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:56.836108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce6a32-8740-4a35-b0c5-860497be272c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:56.837357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 586/1167] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:56.842753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c655e-becb-4d25-923d-19fd87a2a5ee None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:56.843184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c655e-becb-4d25-923d-19fd87a2a5ee None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:56.843318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c655e-becb-4d25-923d-19fd87a2a5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:56.860363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af6c655e-becb-4d25-923d-19fd87a2a5ee None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:56.861527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af6c655e-becb-4d25-923d-19fd87a2a5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:56.925997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 582/1168] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:43:56.948352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6a32-8740-4a35-b0c5-860497be272c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:56.979553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:57.051006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce6a32-8740-4a35-b0c5-860497be272c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:57.084371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:57.085022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:57.085306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:57.085632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:57.086328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:57.087724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd0a55-df12-4cbf-9a0b-c11c73e23aa9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:57.102739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 587/1169] 10.222.0.24 () {60 vars in 1138 bytes} [Tue May 24 13:43:56 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:43:59.315408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c397-22b1-4caa-8837-f204ba4c4007 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:59.315999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c397-22b1-4caa-8837-f204ba4c4007 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:59.315999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769c397-22b1-4caa-8837-f204ba4c4007 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:59.317541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 583/1170] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:43:59.322763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:59.323160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:59.323325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:59.340076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:43:59.341280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:43:59.406482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 588/1171] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:43:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:43:59.429734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769c397-22b1-4caa-8837-f204ba4c4007 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:59.475975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:59.531759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769c397-22b1-4caa-8837-f204ba4c4007 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:59.564007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:59.564858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:59.565142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:59.565465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] PATH_INFO: `/v3/domains/582910c22e84440795d8ac76af61e183` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:59.575863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=582910c22e84440795d8ac76af61e183)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:59.577293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b5a01-7834-42b1-8154-875ee6d63317 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:59.578908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 584/1172] 10.222.0.24 () {56 vars in 1183 bytes} [Tue May 24 13:43:59 2022] GET /identity/v3/domains/582910c22e84440795d8ac76af61e183 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:43:59.585202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:43:59.605376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:43:59.680001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563ca4d-7aa0-42ab-8c31-109fedb3dd4f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:43:59.712696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:43:59.713340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:43:59.713624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:43:59.713898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:43:59.714596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:43:59.716050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8f48e-505c-4621-8e7f-2bab863aa212 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:43:59.739949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 589/1173] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:43:59 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:44:01.959641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c1eb6-937b-4464-bfc6-c02d9483ef5f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:01.960187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c1eb6-937b-4464-bfc6-c02d9483ef5f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:01.960187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c1eb6-937b-4464-bfc6-c02d9483ef5f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:01.961536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 585/1174] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:44:01.966768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:01.967198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:01.967325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:01.984026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:01.985196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:02.049366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 590/1175] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:44:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:44:02.074055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c1eb6-937b-4464-bfc6-c02d9483ef5f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:02.172009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c1eb6-937b-4464-bfc6-c02d9483ef5f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:02.174245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:02.205314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:02.206162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:02.206449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:02.207252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] PATH_INFO: `/v3/domains/582910c22e84440795d8ac76af61e183` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:02.209091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=582910c22e84440795d8ac76af61e183)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:02.210583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396eca6-e423-4a99-81dd-262805a75ade None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:02.212202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 586/1176] 10.222.0.24 () {56 vars in 1183 bytes} [Tue May 24 13:44:02 2022] GET /identity/v3/domains/582910c22e84440795d8ac76af61e183 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:02.217930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:02.222915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:02.317957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e46ae2a-83ab-4298-a8c3-d7ba6da8a574 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:02.350833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:02.351486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:02.351772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:02.352081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:02.352780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:02.354203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e316cc5-1127-4227-a3dc-8c00eb8e084c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:02.372938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 591/1177] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:44:02 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:44:04.613617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3701c-b665-4da3-9702-44a8e565e709 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:04.614135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3701c-b665-4da3-9702-44a8e565e709 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:04.614255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3701c-b665-4da3-9702-44a8e565e709 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:04.615551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 587/1178] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:44:04.620827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:04.621236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:04.621392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:04.640540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:04.641732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:04.693216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:04.707158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 592/1179] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:44:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:44:04.730687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3701c-b665-4da3-9702-44a8e565e709 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:04.833792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3701c-b665-4da3-9702-44a8e565e709 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:04.867621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:04.868458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:04.868746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:04.869025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:04.878944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:44:04.886765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:44:04.887944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:44:04.889054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 588/1180] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:44:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:44:04.894031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:04.989235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834c05b-e1b7-4a4c-b67e-db9190564ee5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:05.022282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:05.022941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:05.023232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:05.023516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:05.024298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:05.024613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:05.025982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:05.033727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 593/1181] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:44:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:05.039662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:05.071263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:05.138550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca587cab-9f09-4c27-95fd-2c5c2a21e747 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:05.170247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:05.171117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:05.171400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:05.171680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] PATH_INFO: `/v3/users/1395411f0c474833b01c0b87283d9815` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:05.184752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] RBAC: Authorizing `identity:get_user(user_id=1395411f0c474833b01c0b87283d9815)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:05.186176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:05.187796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 589/1182] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:44:05 2022] GET /identity/v3/users/1395411f0c474833b01c0b87283d9815 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:05.193824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:05.242566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:05.293547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9373127-d579-4fb1-8445-1227ce334bb1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:05.325847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:05.326479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:05.326766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:05.327044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:05.391178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:05.392596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:05.438835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:05.486962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 594/1183] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:44:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:44:05.492907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:05.580998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:05.591194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1dd0ae-1e46-4ae9-a4b5-bfc163b04392 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:05.623507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:05.624403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:05.624693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:05.624971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] PATH_INFO: `/v3/projects/02ad414bfab944ecb73b77f9bf5d426d` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:05.626140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] RBAC: Authorizing `identity:get_project(project_id=02ad414bfab944ecb73b77f9bf5d426d)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:05.627575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa32960-2758-46d5-aa1c-c63fd8bc65c2 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:05.630108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 590/1184] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:44:05 2022] GET /identity/v3/projects/02ad414bfab944ecb73b77f9bf5d426d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:05.636116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:05.730787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115241-0b95-46f6-93ea-d1157b86fbd7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:05.764709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:05.765355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:05.765641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:05.765920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:05.766774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=02ad414bfab944ecb73b77f9bf5d426d, user.id=1395411f0c474833b01c0b87283d9815) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:05.767094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:05.768539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf66c9-b950-4011-834a-1ec7fe45aa71 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:05.773788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 595/1185] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:44:05 2022] GET /identity/v3/role_assignments?user.id=1395411f0c474833b01c0b87283d9815&scope.project.id=02ad414bfab944ecb73b77f9bf5d426d&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:07.930465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700761c-b9f1-499b-93e4-87bcc1834d43 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:07.930981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700761c-b9f1-499b-93e4-87bcc1834d43 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:07.930981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700761c-b9f1-499b-93e4-87bcc1834d43 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:07.932357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 591/1186] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:44:07.937425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd507-639a-456f-a4e8-806025049816 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:07.937866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd507-639a-456f-a4e8-806025049816 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:07.937968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fddd507-639a-456f-a4e8-806025049816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:07.954747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2fddd507-639a-456f-a4e8-806025049816 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:07.955943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2fddd507-639a-456f-a4e8-806025049816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:07.965409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:08.021287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 596/1187] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:44:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:44:08.043728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700761c-b9f1-499b-93e4-87bcc1834d43 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:08.141658 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700761c-b9f1-499b-93e4-87bcc1834d43 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:08.176570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:08.177362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:08.177652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:08.177933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:08.187792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:44:08.195970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:44:08.197053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:44:08.198080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 592/1188] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:44:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:44:08.203119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddd507-639a-456f-a4e8-806025049816 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:08.298575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fddd507-639a-456f-a4e8-806025049816 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:08.331777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:08.332468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:08.332758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:08.333042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:08.333793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:08.334106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:08.335482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:08.343246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 597/1189] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:44:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:08.350064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:08.369811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:08.448793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade628d9-e949-421e-8e78-800fb9f16a77 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:08.481418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:08.482281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:08.482566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:08.482842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] PATH_INFO: `/v3/users/1395411f0c474833b01c0b87283d9815` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:08.484035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] RBAC: Authorizing `identity:get_user(user_id=1395411f0c474833b01c0b87283d9815)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:08.485453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:08.487053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 593/1190] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:44:08 2022] GET /identity/v3/users/1395411f0c474833b01c0b87283d9815 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:08.493457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:08.502732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:08.591071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21d790-854f-476c-a03e-2b902c9eb97f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:08.624897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:08.625749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:08.626031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:08.626307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] PATH_INFO: `/v3/projects/02ad414bfab944ecb73b77f9bf5d426d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:08.627468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] RBAC: Authorizing `identity:get_project(project_id=02ad414bfab944ecb73b77f9bf5d426d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:08.628949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed108385-56b2-4f10-a5c3-4e3395cd381d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:08.630875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 598/1191] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:44:08 2022] GET /identity/v3/projects/02ad414bfab944ecb73b77f9bf5d426d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:08.636802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:08.719234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:08.734830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff33e6-2c02-48bd-9074-6a7a61099d6e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:08.767065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:08.768011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:08.768298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:08.769094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] PATH_INFO: `/v3/projects/02ad414bfab944ecb73b77f9bf5d426d/users/1395411f0c474833b01c0b87283d9815/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:08.771230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] RBAC: Authorizing `identity:create_grant(project_id=02ad414bfab944ecb73b77f9bf5d426d, user_id=1395411f0c474833b01c0b87283d9815, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:08.772747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a54013-5633-41a2-8a0a-d8e6750b9cbc None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:08.785694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 594/1192] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:44:08 2022] PUT /identity/v3/projects/02ad414bfab944ecb73b77f9bf5d426d/users/1395411f0c474833b01c0b87283d9815/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:44:10.941088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4a2e7-b0f1-4e55-a88f-df89726343f8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:10.941584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4a2e7-b0f1-4e55-a88f-df89726343f8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:10.941678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4a2e7-b0f1-4e55-a88f-df89726343f8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:10.943097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 599/1193] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:44:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:44:10.948553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:10.948976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:10.949125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:10.967173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:44:10.969101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:44:11.034845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 595/1194] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:44:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:44:11.058226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4a2e7-b0f1-4e55-a88f-df89726343f8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.133329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:11.158014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4a2e7-b0f1-4e55-a88f-df89726343f8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:11.190047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:11.191653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:11.192044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:11.192462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:11.202465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:44:11.209926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:44:11.210998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:44:11.211900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 600/1195] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:44:11.217251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.292066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:11.316059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eff21-4ff0-46d9-9301-e5e7ddf8e717 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:11.348937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:11.349603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:11.349894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:11.350175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:11.350934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:11.351249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:11.352648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:11.360435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 596/1196] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:11.366806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.464020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cff08a-8658-4ab4-bd70-543d80fd91c5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:11.495606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:11.496507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:11.496796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:11.497077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] PATH_INFO: `/v3/users/1395411f0c474833b01c0b87283d9815` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:11.498275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] RBAC: Authorizing `identity:get_user(user_id=1395411f0c474833b01c0b87283d9815)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:11.499707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:11.501347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 601/1197] 10.222.0.24 () {56 vars in 1177 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/users/1395411f0c474833b01c0b87283d9815 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:11.507930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.604039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357f48f-2a36-4ed3-81e8-351b4710e4a6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:11.636556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:11.637189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:11.637474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:11.637753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:11.646536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:11.703203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:11.704666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:11.799294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 597/1198] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:44:11.805744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.808614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:11.903509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b312674-0557-45be-a27a-f1c7990912db None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:11.936056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:11.936902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:11.937188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:11.937463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] PATH_INFO: `/v3/projects/02ad414bfab944ecb73b77f9bf5d426d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:11.938627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] RBAC: Authorizing `identity:get_project(project_id=02ad414bfab944ecb73b77f9bf5d426d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:11.940096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98de667-47b3-4d5f-bd51-2555ea3cf413 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:11.941930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 602/1199] 10.222.0.24 () {56 vars in 1186 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/projects/02ad414bfab944ecb73b77f9bf5d426d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:11.947971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:44:11.986683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:12.044850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07911799-0de2-4231-ae5a-94c57a60e214 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:44:12.077278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:44:12.077859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:44:12.078145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:44:12.078421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:44:12.079245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=02ad414bfab944ecb73b77f9bf5d426d, user.id=1395411f0c474833b01c0b87283d9815) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:44:12.079562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:44:12.081683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d2d54-34d2-435a-9cfd-50d8505b7481 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:44:12.087472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 598/1200] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:44:11 2022] GET /identity/v3/role_assignments?user.id=1395411f0c474833b01c0b87283d9815&scope.project.id=02ad414bfab944ecb73b77f9bf5d426d&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:44:21.932099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=65 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:44:22.085335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=49 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:13.019772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60446-b919-4278-b4c6-28c0acb90467 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:13.020380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60446-b919-4278-b4c6-28c0acb90467 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:13.020380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60446-b919-4278-b4c6-28c0acb90467 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:13.022444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 603/1201] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:13.030235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:13.031155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:13.031155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:13.052487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:13.053693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:13.152670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 599/1202] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:45:13.179072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60446-b919-4278-b4c6-28c0acb90467 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:13.330017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60446-b919-4278-b4c6-28c0acb90467 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:13.371543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:13.372590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:13.373304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:13.373959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:13.387668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:13.390463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:13.412964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:13.414011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:13.414970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:13.414970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:13.414970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:13.414970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:13.414970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 604/1203] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:13.421494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:13.561200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76e357-23c7-4048-9163-da8d52e9205d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:13.594420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:13.601066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:13.601778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:13.602132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:13.602418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:13.603191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:13.603726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:13.605121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb943-be8a-4b24-9ee5-86c467516d87 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:13.624620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:13.626895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 600/1204] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:13.694939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:13.755153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b109d3-0e37-4e5e-b99c-cf9f5adc2be7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:13.800619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:13.801456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:13.801866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:13.802281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:13.803094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:13.804782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b0da1-b19a-4390-86cd-e067bd04e5c5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:13.828825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 605/1205] 10.222.0.24 () {60 vars in 1132 bytes} [Tue May 24 13:45:13 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:45:16.255642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ec845-b784-4fb8-adb3-250bd575e03d None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:16.256243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ec845-b784-4fb8-adb3-250bd575e03d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:16.256680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ec845-b784-4fb8-adb3-250bd575e03d None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:16.258609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 601/1206] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:16.270063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:16.270590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:16.270590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:16.293056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:16.294912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:16.377472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 606/1207] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:16.402260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ec845-b784-4fb8-adb3-250bd575e03d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:16.523417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ec845-b784-4fb8-adb3-250bd575e03d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:16.562829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:16.563722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:16.564063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:16.564367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:16.577179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:16.586585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:16.588398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:16.589602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 602/1208] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:45:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:16.594772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:16.644129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:16.716999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f4b93-a23c-45ba-ae76-4714aaf6d211 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:16.758206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:16.758976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:16.759439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:16.759894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:16.760939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:16.761336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:16.763056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:16.771885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 607/1209] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:45:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:16.778128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:16.837283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:16.897330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b5e5e-5134-4ca4-9012-a1c31b4a1a28 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:16.936764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:16.937605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:16.937905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:16.938258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:16.950510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:16.953044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:16.973174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:16.974267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:16.975221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:16.975221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:16.975221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:16.975221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:16.975221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 603/1210] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:16.980695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:17.115717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:17.119063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdd9d2-cc04-4700-980a-844abf1b96bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:17.161949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:17.162802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:17.166107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:17.166107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:17.166107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:17.166107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:17.166107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:17.181138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 608/1211] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:17.188654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:17.310175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e9837-9f2a-4431-bb37-ac7312c9452b None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:17.312216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:17.350246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:17.351068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:17.351360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:17.351645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:17.360522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:17.362136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:17.380050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:17.381064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:17.382026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 604/1212] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:17.387146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:17.520387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d60f34c-969a-41a1-9fc6-347d13d9f0d9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:17.565085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:17.565949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:17.566338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:17.566621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:17.567503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:17.567930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:17.569605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:17.591810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 609/1213] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:17.597606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:17.706523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70667a5b-1c8e-4d98-9af4-34a95804fe27 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:17.746715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:17.747464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:17.748478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:17.748771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:17.780818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:17.829557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:17.831401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:17.946067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 605/1214] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:45:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:45:17.952213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:17.974746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:18.071060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e4d6-2e0f-4db4-a7f6-396d37140794 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:18.111301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:18.112242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:18.112516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:18.112811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:18.125270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:18.127109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:18.146805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:18.147804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:18.148710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:18.148710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:18.148710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:18.148710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:18.154333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:18.155891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 610/1215] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:18.223463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:18.277673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fface134-67b8-4ab4-9872-3aef90fa9210 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:18.316446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:18.317144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:18.317426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:18.317703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:18.318600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:18.318849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:18.320726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:18.333661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 606/1216] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:18.339387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:18.425943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:18.464324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33091f96-61fb-4909-bdb0-f2f9178b473f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:18.514084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:18.514974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:18.515321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:18.515540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:18.524408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:18.526619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:18.544993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:18.546409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:18.547296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 611/1217] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:18.555160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:18.687049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048af3-fd13-4df5-bca3-ab04d93cfa34 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:18.717653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:18.731343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:18.732685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:18.732966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:18.733248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:18.734054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:18.734518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:18.735982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26869bfb-224e-4a71-863e-075dc56614f6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:18.750003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 607/1218] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:18.756569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:18.880487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc5e71-b8d3-4150-b4b0-44cc6dc8d4cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:18.911756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:18.918473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:18.919137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:18.919598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:18.919902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:18.922857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=4121712d68e243f1998602c91d1308d1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:18.922857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:18.922857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a39676-b0cd-4587-a024-485e88955efd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:18.928870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 612/1219] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:45:18 2022] GET /identity/v3/role_assignments?user.id=4121712d68e243f1998602c91d1308d1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:21.249157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424dc-a682-4cac-b062-e19450fabb32 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:21.249776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424dc-a682-4cac-b062-e19450fabb32 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:21.249776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1424dc-a682-4cac-b062-e19450fabb32 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:21.251235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 608/1220] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:21.257007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:21.260446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:21.260446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:21.288977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:21.290311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:21.370823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 613/1221] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 13:45:21.394374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1424dc-a682-4cac-b062-e19450fabb32 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:21.517926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1424dc-a682-4cac-b062-e19450fabb32 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:21.556508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:21.557394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:21.557696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:21.557974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:21.569309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:21.577692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:21.578870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:21.580651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 609/1222] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:45:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:21.585634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:21.697301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5add4-d24f-474f-be3f-026620aaa3e0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:21.724210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:21.735759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:21.736475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:21.736763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:21.737042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:21.737795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:21.738107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:21.739591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:21.750073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 614/1223] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:21.756478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:21.806350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:21.878013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71a1e3-426c-4041-841c-3910ae5cb577 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:21.918014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:21.918857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:21.919268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:21.919682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:21.932773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:21.934263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:21.954751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:21.955979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:21.957063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:21.957063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:21.957063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:21.957063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:21.957063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 610/1224] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:21.963604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.077223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e1fd-4781-478a-b4a9-278708a8e6c8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.124139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.124794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.125142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.125387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.126117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:22.126428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.128166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.153428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 615/1225] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:22.158467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.266279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:22.275287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38bc8ee-78f0-462b-95b4-487e4ad00004 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.319507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.320307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.320593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.320868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.329721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.331056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:22.345838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:22.347018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:22.348045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 611/1226] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:22.353882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.407608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:22.472093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075b9c-005f-4e86-ac3f-1656374d4b3c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.510051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.510749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.511036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.511315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.512194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:22.512511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.515416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.533673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 616/1227] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:22.539338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.659255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586333fb-3d4a-4a6d-8c31-4bae8e36b2fd None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.694911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.695733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.696056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.696356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.707426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.708893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:22.726685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:22.727842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:22.729310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:22.729310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:22.729310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:22.729310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:22.729310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 612/1228] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:22.734615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.847025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8107e-b5c7-460c-b323-c5983ee8bf65 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:22.850628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:22.885391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:22.886081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:22.886388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:22.886696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:22.887385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:22.887686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:22.889505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:22.902142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 617/1229] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:22.908423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:22.918274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:23.022748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43f290b-76bf-45c3-8dd6-819461b6852f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:23.061656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:23.062465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:23.062757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:23.063036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:23.069866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:23.071310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:23.085371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:23.086422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:23.087502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 613/1230] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:23.094077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:23.209886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251c455-e3de-4a0b-9ffb-e1ed15dfc417 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:23.245069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:23.245724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:23.246038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:23.246316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:23.247103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:23.247630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:23.249268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95eeccf-2752-4696-8d0b-c1889c959779 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:23.261769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 618/1231] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:23.269158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:23.339769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:23.380142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f19e5f-df92-4671-bb89-6d169577237f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:23.420698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:23.421743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:23.422125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:23.422506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/4121712d68e243f1998602c91d1308d1/roles/832e4444ee1f4fe3a469481dacbf3b1c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:23.439318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=4121712d68e243f1998602c91d1308d1, role_id=832e4444ee1f4fe3a469481dacbf3b1c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:23.440914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145849c-5540-4dbf-94a9-a9ed1f3df3b7 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:23.454521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 614/1232] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:45:23 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/4121712d68e243f1998602c91d1308d1/roles/832e4444ee1f4fe3a469481dacbf3b1c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:45:25.858930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e84c-39ac-4a69-946b-e2bbc4b4a6cf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:25.859462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e84c-39ac-4a69-946b-e2bbc4b4a6cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:25.859462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4251e84c-39ac-4a69-946b-e2bbc4b4a6cf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:25.860947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 619/1233] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:45:25.866428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:25.866864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:25.867122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:25.884230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:25.885591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:25.958384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 615/1234] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:25.982266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e84c-39ac-4a69-946b-e2bbc4b4a6cf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:26.012175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:26.097821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4251e84c-39ac-4a69-946b-e2bbc4b4a6cf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:26.136952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:26.137773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:26.138078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:26.138343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:26.149805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:26.158331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:26.159454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:26.160358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 620/1235] 10.222.0.24 () {56 vars in 1102 bytes} [Tue May 24 13:45:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:26.166354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:26.219655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:26.286070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32eca32-7dd5-4edc-b68e-c9b03b8f08ba None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:26.324990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:26.325748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:26.326046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:26.326319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:26.327077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] RBAC: Adding query filter params (name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:26.327392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:26.328824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:26.337533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 616/1236] 10.222.0.24 () {56 vars in 1103 bytes} [Tue May 24 13:45:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:26.344708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:26.439334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:26.457054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43407969-e1b4-4689-82d7-4ece47be8f1c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:26.497646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b811823-3122-413b-a413-febdc0149441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:26.498550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b811823-3122-413b-a413-febdc0149441 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:26.498839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b811823-3122-413b-a413-febdc0149441 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:26.499125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b811823-3122-413b-a413-febdc0149441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:26.511465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b811823-3122-413b-a413-febdc0149441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:26.513469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b811823-3122-413b-a413-febdc0149441 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6b811823-3122-413b-a413-febdc0149441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:26.534654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:26.535758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:26.537105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:26.537105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:26.537105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:26.537105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:26.537105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 621/1237] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:26.541847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:26.658278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228309c-29ae-4c5b-ba92-d6c0d02b023c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:26.669912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:26.699586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:26.700304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:26.700599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:26.700867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:26.701646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:26.701945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:26.703310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:26.716383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 617/1238] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:26.722764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b811823-3122-413b-a413-febdc0149441 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:26.829871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b811823-3122-413b-a413-febdc0149441 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:26.864412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:26.865212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:26.865496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:26.865769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:26.868602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:26.874215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:26.875778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:26.891863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:26.893274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:26.894618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 622/1239] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:26.899249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:27.017039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c914be-37b1-48b3-82fa-1a8ce7de731a None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:27.055097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:27.055772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:27.056092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:27.056378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:27.057230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:27.057552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:27.058992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:27.076759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 618/1240] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:27.082899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:27.202472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f45f7f-cddf-496c-81ac-6b1379d501cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:27.246406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:27.247119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:27.247412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:27.247696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:27.334266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:27.342490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:27.344449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:27.454435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 623/1241] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:45:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:45:27.460645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:27.490305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:27.574218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03130b35-040b-4530-a81d-bee88d1b1570 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:27.610376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:27.611102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:27.611384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:27.611659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:27.622847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:27.624365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:27.645770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:27.646762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:27.647624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:27.647624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:27.647624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:27.647624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:27.647624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 619/1242] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:27.654368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:27.761679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:27.769542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b0755-1136-4646-8633-efb557242faa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:27.806968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:27.807632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:27.807948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:27.808226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:27.809031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:27.809518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:27.811201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:27.822864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 624/1243] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:27.832430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:27.920134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:27.947745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb299-69bc-4dff-b14e-ca6ea612b73d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:27.994895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:27.995750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:27.996070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:27.996412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:28.007250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:28.008744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:28.026715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:28.033857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:28.035024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 620/1244] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:28.035975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:28.167354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934a46b-dcdc-408c-b691-3173d30b5f0d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:28.214915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:28.215848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:28.216172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:28.216575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:28.217795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:28.218499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:28.220560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f8263-6402-4c68-a18f-06cdcf77a499 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:28.237238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 625/1245] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:28.243447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:28.382534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a657f-5a42-4235-b800-d93c6881624d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:28.426386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:28.427260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:28.427669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:28.428062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:28.429142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] RBAC: Adding query filter params (role.id=832e4444ee1f4fe3a469481dacbf3b1c, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=4121712d68e243f1998602c91d1308d1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:28.429475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:28.431324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d436f14-1e54-48f7-be04-6be97a802a3e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:28.439369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 621/1246] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:45:28 2022] GET /identity/v3/role_assignments?user.id=4121712d68e243f1998602c91d1308d1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=832e4444ee1f4fe3a469481dacbf3b1c => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:30.838118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18dc3-4f29-47d4-9800-b396a4983ca1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:30.838728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18dc3-4f29-47d4-9800-b396a4983ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:30.838728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe18dc3-4f29-47d4-9800-b396a4983ca1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:30.840237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 626/1247] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:45:30.846814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:30.847360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:30.847360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:30.852571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:30.866515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:30.867717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:30.943171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 622/1248] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:30.969962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe18dc3-4f29-47d4-9800-b396a4983ca1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:30.992218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:31.088548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe18dc3-4f29-47d4-9800-b396a4983ca1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:31.125621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:31.127080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:31.127496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:31.127840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:31.141301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:31.150789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:31.152036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:31.153155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:31.165637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 627/1249] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:45:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:31.165826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:31.282756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597c689-4ab4-42e9-aae8-67ad317aa5f6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:31.319535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:31.327403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:31.328176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:31.328468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:31.328748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:31.329516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:31.329823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:31.331190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:31.341365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 623/1250] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:45:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:31.346909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:31.433708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:31.458431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa25e0-43dc-44c6-aacc-87651001d1c4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:31.493308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:31.494226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:31.494644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:31.495064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:31.509776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:31.512855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:31.540773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:31.541997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:31.543025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:31.543025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:31.543025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:31.543025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:31.543025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 628/1251] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:31.548683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:31.663175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c2cb6-f029-4080-bee5-badcb4df60b4 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:31.701534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:31.702309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:31.702639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:31.702950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:31.703742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:31.704108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:31.705523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:31.718496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 624/1252] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:31.724206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:31.838282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103ccf4-f2d7-4e58-a865-a54b7098c5cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:31.870125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:31.879607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:31.880473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:31.880761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:31.881061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:31.890137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:31.891845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:31.908778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:31.909979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:31.911122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 629/1253] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 13:45:31.916059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:31.979827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:32.027067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1e48a-fe81-404a-b671-ad4148dd75a6 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:32.064684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:32.065383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:32.065670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:32.065949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:32.066813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:32.067126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:32.068652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:32.094537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 625/1254] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:32.100323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:32.217789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81141c53-785c-4584-8f27-6084951f9131 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:32.252475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:32.253122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:32.253420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:32.253693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:32.319577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:32.333189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:32.334695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:32.437556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 630/1255] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:45:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:45:32.443428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:32.550389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3414f9-2cb8-4a46-ac45-ea676481c39e None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:32.564294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:32.587142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:32.587979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:32.588272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:32.588555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:32.599147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:32.600640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:32.618605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:32.619681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:32.620940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:32.620940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:32.620940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:32.620940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:32.620940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 626/1256] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:32.627374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:32.715863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:32.727996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62def0-8780-4d76-adc2-69d57f6fc645 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:32.760551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:32.761153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:32.761448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:32.761728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:32.762967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:32.763279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:32.764678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:32.775531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 631/1257] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:32.781334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:32.878484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691ce7e-5512-496f-a478-ed4cbb858b83 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:32.910828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:32.911602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:32.911914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:32.912193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:32.919087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:32.920538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:32.933714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:32.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:32.935948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 627/1258] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:32.940608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:33.037477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee915d1-baa2-47d0-99b0-2e285d3cfb80 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:33.068991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:33.069641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:33.069932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:33.070216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:33.071003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:33.071318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:33.072790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621f11-8dfc-4198-80e5-556607456b0c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:33.084805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 632/1259] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:33.091548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:33.116681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:33.189702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4b711-8a23-4296-8a09-f0789a2cf18d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:33.221934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:33.222571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:33.222855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:33.223133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:33.223988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=4121712d68e243f1998602c91d1308d1) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:33.224305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:33.225729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e860f58-9a12-40e7-96c0-bb2e0f5d06ce None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:33.231381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 628/1260] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:45:33 2022] GET /identity/v3/role_assignments?user.id=4121712d68e243f1998602c91d1308d1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:35.406554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225b2d-04be-424d-ba16-a9a23c8bc8a9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.407111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225b2d-04be-424d-ba16-a9a23c8bc8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.407111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99225b2d-04be-424d-ba16-a9a23c8bc8a9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.408483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 633/1261] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:35.413684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.414113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.414256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.431418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:35.432804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:35.498493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 629/1262] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:35.521430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225b2d-04be-424d-ba16-a9a23c8bc8a9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:35.532487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:35.619878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99225b2d-04be-424d-ba16-a9a23c8bc8a9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:35.653414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:35.654194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.654479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.654756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.664386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:35.672101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:35.673184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:35.674084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 634/1263] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:45:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:35.679074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:35.697309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:35.776403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28add6fd-4500-4dc5-ae16-4fff41a31496 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:35.809535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:35.810187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.810473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.810750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.811495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:35.811804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:35.813235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:35.821091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 630/1264] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:45:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:35.827008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:35.891557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:35.924989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e91b70-adc6-480d-a397-c8caad59e805 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:35.957499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:35.958279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:35.958569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:35.958851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:35.969059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:35.970515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:35.987540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:35.988595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:35.989616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:35.989616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:35.989616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:35.989616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:35.989616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 635/1265] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:35.994484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.073797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:36.093493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787149e-9298-4969-a4e0-60fd9b10bc38 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.125780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.126694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.126978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.127256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.128003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:36.128312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.129667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.141703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 631/1266] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:36.147247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.242390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2983e-fa12-40ac-8a62-c994cfc70824 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.264128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:36.279309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.280139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.280427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.280707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.289048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.290547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:36.306647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:36.307662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:36.309133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 636/1267] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:36.314566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.416736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8ca32-7aca-48ee-8e4b-cdf98fb64612 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.450857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.451547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.451862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.452149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.452987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:36.453303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.454718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.463187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:36.471841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 632/1268] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:36.477374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.576540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7f1a7-1fea-4640-b433-525d3e0432c4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.608749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.609525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.609808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.610083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.620085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.622386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:36.639274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:36.640346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:36.641604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:36.641604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:36.641604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:36.641604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:36.641604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 637/1269] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:36.646060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.741804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cb0bd7-03f3-493e-897a-b84b635da21c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.775402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.776027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.776320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.776602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.777358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:36.777673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.779046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.790017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 633/1270] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:36.796156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:36.827439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:36.894634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275691b-d1de-458a-b19c-5d58ac414b24 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:36.927517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:36.928324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:36.928610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:36.928885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:36.935685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:36.937151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:36.951159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:36.952249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:36.953373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 638/1271] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:36.958033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:37.015165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:37.056014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f4f54-e84c-484c-a826-70ec24e88ea9 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:37.087651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:37.088330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:37.088613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:37.088886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:37.089670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:37.090489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:37.091927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a1eec-54b5-4175-8c99-c0cd5af4ee85 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:37.104211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 634/1272] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:37.113235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:37.204768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:37.213597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e17f61-36af-490b-bd2f-df3223d77ed6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:37.247384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:37.248290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:37.248572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:37.248846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] PATH_INFO: `/v3/projects/bff552f050b044b598ad782ed83d42c7/users/4121712d68e243f1998602c91d1308d1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:37.250773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] RBAC: Authorizing `identity:create_grant(project_id=bff552f050b044b598ad782ed83d42c7, user_id=4121712d68e243f1998602c91d1308d1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:37.252534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5c5de-a0aa-40bf-9473-bf70466e51db None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:37.266221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 639/1273] 10.222.0.24 () {58 vars in 1439 bytes} [Tue May 24 13:45:37 2022] PUT /identity/v3/projects/bff552f050b044b598ad782ed83d42c7/users/4121712d68e243f1998602c91d1308d1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:45:39.482674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7e912-ffa1-43b6-b549-8bfd2f760894 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:39.483204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7e912-ffa1-43b6-b549-8bfd2f760894 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:39.483204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d7e912-ffa1-43b6-b549-8bfd2f760894 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:39.484628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 635/1274] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:39.489825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:39.490241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:39.490393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:39.507986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:39.509187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:39.572802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 640/1275] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:39.600613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7e912-ffa1-43b6-b549-8bfd2f760894 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:39.698356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d7e912-ffa1-43b6-b549-8bfd2f760894 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:39.716619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:39.733598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:39.734323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:39.734605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:39.734880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:39.744548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 13:45:39.753279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:39.754351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:39.755287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 636/1276] 10.222.0.24 () {56 vars in 1096 bytes} [Tue May 24 13:45:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:39.760377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:39.839034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:39.858789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126260d-4063-4235-b750-e9c34adbcf1c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:39.891392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:39.892093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:39.892385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:39.892668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:39.893415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:39.893727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:39.895107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:39.903424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 641/1277] 10.222.0.24 () {56 vars in 1099 bytes} [Tue May 24 13:45:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:39.909068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.006357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7adf90-102c-4fb3-99a8-81088f55550c None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:40.038797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:40.039570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:40.039882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:40.040165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:40.051040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:40.052503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:40.069171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:40.070233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:40.071145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:40.071145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:40.071145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:40.071145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:40.071145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 637/1278] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:40.076051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.175031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa458af-9d31-4b8b-a5dc-805788a854a1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:40.207592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:40.208264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:40.208552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:40.208830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:40.209544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:40.209854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:40.211216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:40.222732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 642/1279] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:40.228393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.261974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:40.337177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771c95-98fd-4e66-a64e-6bfbc6996e02 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:40.369545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:40.370338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:40.370623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:40.371645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:40.379130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:40.380594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 13:45:40.394615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:40.395712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:40.396908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 638/1280] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:40.401640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.405591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:40.499416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b133fc3-46f3-462e-92b8-dfc271393c40 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:40.532659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:40.533275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:40.533564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:40.533847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:40.534683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:40.535000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:40.536464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:40.553398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 643/1281] 10.222.0.24 () {56 vars in 1143 bytes} [Tue May 24 13:45:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:40.559847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.649675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:40.659951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6af61a-ac46-48c0-8753-9176dfd08a37 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:40.692872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:40.693510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:40.693800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:40.694079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:40.766855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:40.768326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:40.853745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:40.864019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 639/1282] 10.222.0.24 () {58 vars in 1282 bytes} [Tue May 24 13:45:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 13:45:40.869828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:40.930160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:40.969690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51922799-ab09-4aac-b0ca-004ef4ce926a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:41.003560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:41.004371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:41.004658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:41.004933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:41.015439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:41.016918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 13:45:41.034582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:41.035620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:41.036766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 13:45:41.036766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 13:45:41.036766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 13:45:41.036766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:41.036766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 644/1283] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:45:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:41.041506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:41.138157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f317d2b-6ff9-465e-a5f3-199cd58dd9b0 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:41.171252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:41.171939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:41.172232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:41.172519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:41.173249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:41.173564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:41.174937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:41.186109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 640/1284] 10.222.0.24 () {56 vars in 1109 bytes} [Tue May 24 13:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:41.191920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:41.291042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e7ef35-c490-4658-be88-4e3835629f40 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:41.304552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:41.325142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:41.325929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:41.326218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:41.326498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:41.332975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:41.334461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 13:45:41.348446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:41.349482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 24 13:45:41.355595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 645/1285] 10.222.0.24 () {56 vars in 1111 bytes} [Tue May 24 13:45:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:41.356278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:41.405025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:41.454252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0dfd7-00e8-4635-b7fc-01ed0f96d02f None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:41.486240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:41.486890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:41.487173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:41.487448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:41.488262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:41.488576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:41.489995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb0c8a-689c-4f2b-98ff-be6a1d2a3484 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:41.502590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 641/1286] 10.222.0.24 () {56 vars in 1148 bytes} [Tue May 24 13:45:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:41.508555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:41.605301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55f968-fb45-4718-bbf3-805b58204fd2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:41.638473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:41.639117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:41.639408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:41.639693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:41.640557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] RBAC: Adding query filter params (role.id=25fbe1911e9347e3b1369ba1ec4e04ea, scope.project.id=bff552f050b044b598ad782ed83d42c7, user.id=4121712d68e243f1998602c91d1308d1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:41.640874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:41.642304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cb8ad-c023-4021-afc6-d750639aad28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:41.648024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 646/1287] 10.222.0.24 () {56 vars in 1374 bytes} [Tue May 24 13:45:41 2022] GET /identity/v3/role_assignments?user.id=4121712d68e243f1998602c91d1308d1&scope.project.id=bff552f050b044b598ad782ed83d42c7&role.id=25fbe1911e9347e3b1369ba1ec4e04ea => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:43.842742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe83dc-f770-474e-87c5-5c8bdedfe734 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:43.843241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe83dc-f770-474e-87c5-5c8bdedfe734 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:43.843241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe83dc-f770-474e-87c5-5c8bdedfe734 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:43.844625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 642/1288] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:43.849805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:43.850229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:43.850373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:43.866702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:43.867920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:43.897501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:43.933932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 647/1289] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:43.956444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe83dc-f770-474e-87c5-5c8bdedfe734 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:44.053997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe83dc-f770-474e-87c5-5c8bdedfe734 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:44.065113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:44.087908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:44.088686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:44.088973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:44.089283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:44.089928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:44.091281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:44.102826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:44.104066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:44.105861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 643/1290] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:45:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:44.110505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:44.206603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43f9b5e-5f5d-4158-9011-9edd3b8fc677 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:44.239455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:44.240135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:44.240424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:44.240705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:44.241410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:44.241722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:44.243089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5f60b-5e63-456a-ab70-ad6049d18e2e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:44.248377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 648/1291] 10.222.0.24 () {56 vars in 1116 bytes} [Tue May 24 13:45:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:44.254576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:44.364927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451182a1-069e-499b-a468-edd1258d9831 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:44.397112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:44.397773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:44.398064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:44.398348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] PATH_INFO: `/v3/services` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:44.399069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:44.399388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:44.400790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c7206d-f745-4a64-9878-e56124a17d64 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:44.405813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 644/1292] 10.222.0.24 () {56 vars in 1116 bytes} [Tue May 24 13:45:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:46.596593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3048ea-0e43-4af8-925d-2f3e22a2e078 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:46.597142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3048ea-0e43-4af8-925d-2f3e22a2e078 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:46.597142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3048ea-0e43-4af8-925d-2f3e22a2e078 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:46.598486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 649/1293] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:46.603653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a0e76-eb60-4003-a2a3-a8d7cfaf9c3a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:46.604093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a0e76-eb60-4003-a2a3-a8d7cfaf9c3a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:46.604265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2a0e76-eb60-4003-a2a3-a8d7cfaf9c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:46.621813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6d2a0e76-eb60-4003-a2a3-a8d7cfaf9c3a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:46.622984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6d2a0e76-eb60-4003-a2a3-a8d7cfaf9c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:46.651318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:46.689834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 645/1294] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:46.711459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3048ea-0e43-4af8-925d-2f3e22a2e078 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:46.742594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:46.813465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3048ea-0e43-4af8-925d-2f3e22a2e078 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:46.851640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:46.852341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:46.852632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:46.852917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:46.853641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:46.855677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29dafc0-c119-4159-89c5-0d35cb5a8819 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:46.868249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 650/1295] 10.222.0.24 () {60 vars in 1141 bytes} [Tue May 24 13:45:46 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:45:49.057306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb6cae-4511-4b79-a612-04d63db20bfc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.058501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb6cae-4511-4b79-a612-04d63db20bfc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.058501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb6cae-4511-4b79-a612-04d63db20bfc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.060631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 646/1296] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:49.067677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.068196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.068456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.086356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:49.087524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:49.158137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 651/1297] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:49.180468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb6cae-4511-4b79-a612-04d63db20bfc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:49.286351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb6cae-4511-4b79-a612-04d63db20bfc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:49.321593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:49.322323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.322605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.322882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.323605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:49.325007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:49.336642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:49.337803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:49.343767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 647/1298] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:45:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:49.344772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:49.362337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:49.440186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aea51d-1e3e-4491-a36b-cfbf8c7db4d3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:49.471827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:49.472498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.472784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.473062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.473759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:49.474065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:49.475423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:49.480891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 652/1299] 10.222.0.24 () {56 vars in 1116 bytes} [Tue May 24 13:45:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:49.486673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:49.518287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:49.584922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b04ae2-762a-4219-af0e-91b0a4a3784d None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:49.617748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:49.618395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.618683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.618962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.619749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] RBAC: Adding query filter params (interface=public, service_id=c0df925a2a2e4c92a56f58d1a6b2abbc, region_id=RegionOne) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:49.620237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:49.621577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea576f-4c35-4ed0-9d4a-1d9b4f29dfec None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:49.626728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 648/1300] 10.222.0.24 () {56 vars in 1251 bytes} [Tue May 24 13:45:49 2022] GET /identity/v3/endpoints?service_id=c0df925a2a2e4c92a56f58d1a6b2abbc&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:49.632296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:49.711187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:49.730232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d3d26-7330-4275-ba63-b9030c420022 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:49.762954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:49.763608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:49.763923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:49.764205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:49.764891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:49.766262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2b66a-5c6b-4c71-9df0-fb76e118c227 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:49.772841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 653/1301] 10.222.0.24 () {56 vars in 1087 bytes} [Tue May 24 13:45:49 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:45:51.955404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a9bd-5bb1-465c-abdd-30b3b87ba3c8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:51.955957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a9bd-5bb1-465c-abdd-30b3b87ba3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:51.955957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4a9bd-5bb1-465c-abdd-30b3b87ba3c8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:51.957301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 649/1302] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:45:51.962298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:51.962670 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:51.962857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:51.980195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:45:51.981368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:45:52.046902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 654/1303] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:45:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 13:45:52.073935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4a9bd-5bb1-465c-abdd-30b3b87ba3c8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:52.165425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:52.172317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4a9bd-5bb1-465c-abdd-30b3b87ba3c8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:52.204664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:52.205501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:52.205740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:52.206020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:52.206673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:52.208072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 13:45:52.219607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 13:45:52.220757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.get_or_create( May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application with Lock( May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self._enter() May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application return self.creator() May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application created_value = creator( May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: ERROR keystone.server.flask.application May 24 13:45:52.221619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 650/1304] 10.222.0.24 () {56 vars in 1117 bytes} [Tue May 24 13:45:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 13:45:52.226769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:52.301075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:45:52.324337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826294c1-643a-4dba-a47f-1ca8320bf83d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:52.357318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:52.357970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:52.358260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:52.358542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:52.359244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:45:52.359554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:52.360959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bb849-58bb-4f26-b371-4bbfea8f2e08 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:52.365952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 655/1305] 10.222.0.24 () {56 vars in 1116 bytes} [Tue May 24 13:45:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:45:52.372143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:45:52.474769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98922ae9-8c77-4816-9238-3023ef598888 None admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:45:52.508867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:45:52.509527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:45:52.509819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:45:52.510097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:45:52.510876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:45:52.512280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e753c7-c9a7-4246-bc0e-4de1e509d6d8 None admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:45:52.529810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 651/1306] 10.222.0.24 () {60 vars in 1144 bytes} [Tue May 24 13:45:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:46:02.363505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=30 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:46:02.526081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:46:56.744075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090731-7c7a-4be3-bd2b-8bba4167a4db None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:46:56.744637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090731-7c7a-4be3-bd2b-8bba4167a4db None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:46:56.744637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae090731-7c7a-4be3-bd2b-8bba4167a4db None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:46:56.747672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 656/1307] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:46:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:46:56.751525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8f385-2ffb-486e-b9b2-0a524f8c59d3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:46:56.751959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8f385-2ffb-486e-b9b2-0a524f8c59d3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:46:56.752185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8f385-2ffb-486e-b9b2-0a524f8c59d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:46:56.853929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a3b8f385-2ffb-486e-b9b2-0a524f8c59d3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:46:56.855170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a3b8f385-2ffb-486e-b9b2-0a524f8c59d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:46:56.984357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 652/1308] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:46:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:46:57.038321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:46:57.038522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:46:57.038810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:46:57.040460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 657/1309] 10.222.0.24 () {54 vars in 964 bytes} [Tue May 24 13:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:46:57.053332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e5c9-53fe-4fb9-9d63-41441bed9040 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:46:57.053630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e5c9-53fe-4fb9-9d63-41441bed9040 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:46:57.053926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3e5c9-53fe-4fb9-9d63-41441bed9040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:46:57.215370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d3a3e5c9-53fe-4fb9-9d63-41441bed9040 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:46:57.218052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d3a3e5c9-53fe-4fb9-9d63-41441bed9040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:46:57.270569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:46:57.359137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:46:57.360823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:46:57.372288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 653/1310] 10.222.0.24 () {58 vars in 1061 bytes} [Tue May 24 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:46:57.498194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b00932-f0fd-4e76-b5e4-bf2dc6b222bf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:46:57.581470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:46:57.582139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:46:57.582426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:46:57.582703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:46:57.585357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:46:57.646592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:46:57.705216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:46:57.706779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d2b93-549f-4c48-8ecd-eade96726bd8 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:46:57.728448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 658/1311] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:47:00.360037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:00.360468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:00.360598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:00.366691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 654/1312] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:00.369489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580b11-891b-4d3f-930e-af2143fed200 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:00.369903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580b11-891b-4d3f-930e-af2143fed200 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:00.370056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b580b11-891b-4d3f-930e-af2143fed200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:00.444503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3b580b11-891b-4d3f-930e-af2143fed200 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:00.445725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3b580b11-891b-4d3f-930e-af2143fed200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:00.555616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 659/1313] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:00.592699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:00.594251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:00.697917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5059a-0db1-4976-b33f-f1cac3b39319 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:00.769888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:00.773396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:00.774101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:00.774639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:00.775069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:00.776966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:00.888255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:00.889702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea849c-eaa5-487d-ac13-7aac5d79ff2b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:00.908117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 655/1314] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:47:03.451439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:03.452063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:03.452063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:03.453374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 660/1315] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:03.458701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d970-7d3a-485c-81d0-19fb3a225d8b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:03.459120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d970-7d3a-485c-81d0-19fb3a225d8b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:03.459256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935d970-7d3a-485c-81d0-19fb3a225d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:03.528015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b935d970-7d3a-485c-81d0-19fb3a225d8b None None] MFA Rules not processed for user `bf9b7c7fda6947388c02a47587519de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:03.529237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b935d970-7d3a-485c-81d0-19fb3a225d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:03.581491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:03.625954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 656/1316] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:03.664923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:03.666484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:03.676041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:03.790809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5b667-531d-495c-a4fc-32af7853596c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:03.868899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:03.869554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:03.869851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:03.870131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:03.871792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:03.933167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:03.979887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:03.981328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb98b5-def7-4945-8273-8d18f8d8428e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:03.999564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 661/1317] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:06.801843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:06.802361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:06.802361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:06.803706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 657/1318] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:06.809092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020054ae-d829-4597-9c65-a39d8bb464b5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:06.809508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020054ae-d829-4597-9c65-a39d8bb464b5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:06.809646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020054ae-d829-4597-9c65-a39d8bb464b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:06.873895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-020054ae-d829-4597-9c65-a39d8bb464b5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:06.875074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-020054ae-d829-4597-9c65-a39d8bb464b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:06.969366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 662/1319] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:07.010003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:07.011528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:07.111916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe1fdd-9a38-4468-b433-914d8af32f15 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:07.184487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:07.185127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:07.185412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:07.185688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:07.187305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:07.195583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:07.296388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:07.297838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9f961-1944-4d15-b28b-0521e11ebca6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:07.315663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 658/1320] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:10.182727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:10.183261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:10.183261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:10.184845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 663/1321] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:10.189752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4efb-7617-426c-9caa-de88e96241e6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:10.190169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4efb-7617-426c-9caa-de88e96241e6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:10.190358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49a4efb-7617-426c-9caa-de88e96241e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:10.254488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b49a4efb-7617-426c-9caa-de88e96241e6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:10.255695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b49a4efb-7617-426c-9caa-de88e96241e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:10.312621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:10.362995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 659/1322] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:10.398935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:10.400525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:10.420461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:10.511660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ef8c8-1fd1-4868-a474-19919352ab76 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:10.586575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:10.587230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:10.587516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:10.587794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:10.589482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:10.663463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:10.702720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:10.704193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457fa7e-c64d-4f69-93c0-72114969f22b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:10.723904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 664/1323] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:13.025993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48756c56-f4d0-4458-a149-df483b9a1424 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:13.026485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48756c56-f4d0-4458-a149-df483b9a1424 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:13.026580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48756c56-f4d0-4458-a149-df483b9a1424 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:13.029115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 660/1324] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:13.033260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910330-d9f1-491f-9810-6b57cd24cdd2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:13.033676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910330-d9f1-491f-9810-6b57cd24cdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:13.033827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6910330-d9f1-491f-9810-6b57cd24cdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:13.099296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c6910330-d9f1-491f-9810-6b57cd24cdd2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:13.100538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6910330-d9f1-491f-9810-6b57cd24cdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:13.196048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 665/1325] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:13.244632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:13.244936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:13.245292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:13.247017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 661/1326] 10.222.0.24 () {54 vars in 964 bytes} [Tue May 24 13:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:13.252841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef05-6626-4cc8-abd4-3f1e435f1640 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:13.253134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef05-6626-4cc8-abd4-3f1e435f1640 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:13.253417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7ef05-6626-4cc8-abd4-3f1e435f1640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:13.286241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:13.330002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5ad7ef05-6626-4cc8-abd4-3f1e435f1640 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:13.331207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5ad7ef05-6626-4cc8-abd4-3f1e435f1640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:13.406091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 666/1327] 10.222.0.24 () {58 vars in 1061 bytes} [Tue May 24 13:47:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:47:13.415221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:13.416783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:13.522415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace9d3f-b860-4ebf-9ac2-c0d666182ec9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:13.599475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:13.600158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:13.600448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:13.600726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:13.602369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:13.619583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:13.712012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:13.713449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e38c13-3e8e-4ab9-ae7e-d89655bb537d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:13.731103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 662/1328] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:16.225261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:16.225827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:16.225827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:16.227205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 667/1329] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:16.232832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1cdc-4bb1-4752-82ad-b4e8facb49d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:16.233238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1cdc-4bb1-4752-82ad-b4e8facb49d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:16.233397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea1cdc-4bb1-4752-82ad-b4e8facb49d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:16.298015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a9ea1cdc-4bb1-4752-82ad-b4e8facb49d1 None None] MFA Rules not processed for user `bf9b7c7fda6947388c02a47587519de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:16.299211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a9ea1cdc-4bb1-4752-82ad-b4e8facb49d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:16.364205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:16.398377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 663/1330] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:16.435351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:16.437054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:16.546120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd6d3e-f099-49ef-8b01-c540317251e2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:16.566382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:16.620765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:16.621664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:16.621957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:16.622237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:16.623920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:16.730754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:16.732245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35dfae-11d9-4592-8fac-3fb6b50c2e39 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:16.750777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 668/1331] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:19.402699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:19.403206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:19.403301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:19.404683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 664/1332] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:19.410751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d486e-a8ef-471d-b78b-82f00428d759 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:19.411215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d486e-a8ef-471d-b78b-82f00428d759 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:19.411319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3d486e-a8ef-471d-b78b-82f00428d759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:19.478514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:19.482007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c3d486e-a8ef-471d-b78b-82f00428d759 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:19.483613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c3d486e-a8ef-471d-b78b-82f00428d759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:19.595356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 669/1333] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:19.631184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:19.632818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:19.735276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e380f3a2-52ea-4596-88bb-0953b0289e7b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:19.807802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:19.812363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:19.812701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:19.813112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:19.815366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:19.844396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:19.926804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:19.928276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3574a2a1-efd9-43e3-8ca3-6c7edf12e883 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:19.946641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 665/1334] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:22.199939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:22.207446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:22.207446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:22.207446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 670/1335] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:22.208294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37577a-c9da-4779-b660-cb16ba8f95a7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:22.208718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37577a-c9da-4779-b660-cb16ba8f95a7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:22.208858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37577a-c9da-4779-b660-cb16ba8f95a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:22.272362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4a37577a-c9da-4779-b660-cb16ba8f95a7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:22.273551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4a37577a-c9da-4779-b660-cb16ba8f95a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:22.346454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:22.369809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 666/1336] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:22.409730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:22.411520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:22.514711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88757c09-b5a9-4e0c-83aa-5de7d42e2a23 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:22.543075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:22.588163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:22.588809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:22.589093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:22.589371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:22.591042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:22.697128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:22.698563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ad588-0fe0-406a-a7ef-4479ba02b1f3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:22.716542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 671/1337] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:25.258920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:25.259437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:25.259437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:25.261018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 667/1338] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:25.266664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c794b8-e980-42a7-8f3f-c46aae0c70ed None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:25.267409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c794b8-e980-42a7-8f3f-c46aae0c70ed None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:25.267409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c794b8-e980-42a7-8f3f-c46aae0c70ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:25.332034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5c794b8-e980-42a7-8f3f-c46aae0c70ed None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:25.333234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5c794b8-e980-42a7-8f3f-c46aae0c70ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:25.341268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:25.429806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 672/1339] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:47:25.467716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:25.469311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:25.571740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14980d26-83ba-4b25-b4ad-5cc014adcd58 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:25.646401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:25.647052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:25.647341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:25.647619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:25.649288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:25.684048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:25.755670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:25.757149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf8e3b1-fb39-4415-97ca-6e7a442bb7dc service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:25.782458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 668/1340] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:30.019310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:30.019865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:30.019865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:30.021205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 673/1341] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:30.027415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285daa7-07d1-46ee-bda6-61c04e9b1aca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:30.027978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285daa7-07d1-46ee-bda6-61c04e9b1aca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:30.028487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285daa7-07d1-46ee-bda6-61c04e9b1aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:30.124447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9285daa7-07d1-46ee-bda6-61c04e9b1aca None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:30.126194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9285daa7-07d1-46ee-bda6-61c04e9b1aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:30.239963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:30.255035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 669/1342] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:30.294338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:30.296764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:30.417067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb5b22-3d9b-426a-95e9-fa5362d5438a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:30.420746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:30.421412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:30.421709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:30.421991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:30.433257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:30.434708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4d12e-5c9a-4d99-8656-e634e6fb8807 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:30.447502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 674/1343] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:33.466770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937c97-0427-4bb9-96f2-48693c96e3a6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:33.467333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937c97-0427-4bb9-96f2-48693c96e3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:33.467333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb937c97-0427-4bb9-96f2-48693c96e3a6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:33.468748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 670/1344] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:33.474581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b7dde-53f4-4ff9-9478-14aabdafbcd9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:33.475022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b7dde-53f4-4ff9-9478-14aabdafbcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:33.475206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b7dde-53f4-4ff9-9478-14aabdafbcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:33.507628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-046b7dde-53f4-4ff9-9478-14aabdafbcd9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:33.508941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-046b7dde-53f4-4ff9-9478-14aabdafbcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:33.537474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:33.586353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 675/1345] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:33.623912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb937c97-0427-4bb9-96f2-48693c96e3a6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:33.637341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb937c97-0427-4bb9-96f2-48693c96e3a6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:33.641088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:33.641734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:33.642047 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:33.642321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:33.652549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:33.654002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2e1e9-c665-4a4c-ae57-75525861d162 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:33.663633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 671/1346] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:38.041316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2fe66-349d-4a28-9867-21f625c9fa6c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:38.041812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2fe66-349d-4a28-9867-21f625c9fa6c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:38.041812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2fe66-349d-4a28-9867-21f625c9fa6c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:38.043162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 676/1347] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:38.048331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ccb6-c6eb-4cdf-90bf-d152bb6d654e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:38.048759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ccb6-c6eb-4cdf-90bf-d152bb6d654e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:38.048915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5ccb6-c6eb-4cdf-90bf-d152bb6d654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:38.068248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9ff5ccb6-c6eb-4cdf-90bf-d152bb6d654e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:38.069388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9ff5ccb6-c6eb-4cdf-90bf-d152bb6d654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:38.077397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 672/1348] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:38.115488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2fe66-349d-4a28-9867-21f625c9fa6c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:38.126150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2fe66-349d-4a28-9867-21f625c9fa6c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:38.129646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:38.130273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:38.130557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:38.130833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:38.139466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:38.140919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a801a-3941-4251-b192-f201953359df service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:38.150384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 677/1349] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:43.612770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e801-536f-4c1e-acb9-20c17cb31848 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:43.614472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e801-536f-4c1e-acb9-20c17cb31848 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:43.614960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280e801-536f-4c1e-acb9-20c17cb31848 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:43.616785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 673/1350] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:43.621868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb87bb-9c7c-4fdb-afdc-d7d8b6fcc05b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:43.622696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb87bb-9c7c-4fdb-afdc-d7d8b6fcc05b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:43.623466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb87bb-9c7c-4fdb-afdc-d7d8b6fcc05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:43.700241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8aeb87bb-9c7c-4fdb-afdc-d7d8b6fcc05b None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:43.701457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8aeb87bb-9c7c-4fdb-afdc-d7d8b6fcc05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:43.788661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 678/1351] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:47:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:48.049733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701314-7135-4c9d-8f23-57bb35892d2f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:48.050614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701314-7135-4c9d-8f23-57bb35892d2f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:48.051503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96701314-7135-4c9d-8f23-57bb35892d2f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:48.052943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 674/1352] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:48.058364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b97777-102e-4aa5-b83a-5561c63afd0e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:48.058823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b97777-102e-4aa5-b83a-5561c63afd0e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:48.059018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b97777-102e-4aa5-b83a-5561c63afd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:48.064186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=13 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:48.078883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-43b97777-102e-4aa5-b83a-5561c63afd0e None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:48.080273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-43b97777-102e-4aa5-b83a-5561c63afd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:48.092140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 679/1353] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:47:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:49.509708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce218-64eb-4588-8fe6-d47782893a87 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:49.510181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce218-64eb-4588-8fe6-d47782893a87 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:49.510278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbce218-64eb-4588-8fe6-d47782893a87 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:49.511642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 675/1354] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:49.516650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0c0-1240-4c13-9a10-259740c32fd6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:49.517064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0c0-1240-4c13-9a10-259740c32fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:49.517217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0c0-1240-4c13-9a10-259740c32fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:49.536266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-eb37f0c0-1240-4c13-9a10-259740c32fd6 None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:49.537646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-eb37f0c0-1240-4c13-9a10-259740c32fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:49.546281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 680/1355] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:47:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:53.619927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391000f1-67c1-4e0c-b6bb-3b5f95507e22 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:53.620429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391000f1-67c1-4e0c-b6bb-3b5f95507e22 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:53.620513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391000f1-67c1-4e0c-b6bb-3b5f95507e22 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:53.622583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 676/1356] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:53.627753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbf-e01f-4f2f-b8ec-d138ae898147 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:53.628175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbf-e01f-4f2f-b8ec-d138ae898147 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:53.628383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dcbf-e01f-4f2f-b8ec-d138ae898147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:53.647444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4497dcbf-e01f-4f2f-b8ec-d138ae898147 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:53.648900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4497dcbf-e01f-4f2f-b8ec-d138ae898147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:53.658109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 681/1357] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:53.713606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8dfe8-f0f5-4936-8ed3-52e0c088db75 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:53.713908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8dfe8-f0f5-4936-8ed3-52e0c088db75 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:53.714213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf8dfe8-f0f5-4936-8ed3-52e0c088db75 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:53.716367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 677/1358] 10.222.0.24 () {54 vars in 961 bytes} [Tue May 24 13:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:53.727184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845f72-30d6-4771-a62d-223186fc1870 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:53.728115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845f72-30d6-4771-a62d-223186fc1870 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:53.728798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f845f72-30d6-4771-a62d-223186fc1870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:53.812869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:47:53.822203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1f845f72-30d6-4771-a62d-223186fc1870 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:53.823708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f845f72-30d6-4771-a62d-223186fc1870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:53.915865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 682/1359] 10.222.0.24 () {58 vars in 1058 bytes} [Tue May 24 13:47:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:53.927538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8dfe8-f0f5-4936-8ed3-52e0c088db75 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:53.943490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf8dfe8-f0f5-4936-8ed3-52e0c088db75 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:53.948994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:53.949666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:53.949952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:53.950408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:53.960430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:53.962034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ca08c-2d64-4eb2-ac7b-6482f2d3cb03 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:53.971972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 678/1360] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:56.446422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3c368-af10-4235-8a1c-c2240ee515af None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:56.446914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3c368-af10-4235-8a1c-c2240ee515af None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:56.447033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3c368-af10-4235-8a1c-c2240ee515af None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:56.448392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 683/1361] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:56.453683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d151-525b-455a-afcc-bef92347b97e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:56.454161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d151-525b-455a-afcc-bef92347b97e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:56.454265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e2d151-525b-455a-afcc-bef92347b97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:56.474309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-86e2d151-525b-455a-afcc-bef92347b97e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:56.475850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-86e2d151-525b-455a-afcc-bef92347b97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:56.484992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 679/1362] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:56.531259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3c368-af10-4235-8a1c-c2240ee515af None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:56.546205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3c368-af10-4235-8a1c-c2240ee515af None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:56.552895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:56.553648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:56.554047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:56.554437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:56.563008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:56.564757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5961d418-c479-4487-b9a6-d3f78f837a3f service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:56.575454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 684/1363] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:47:59.134954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aab4-2604-4276-8279-cfb01de77318 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:59.136012 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aab4-2604-4276-8279-cfb01de77318 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:59.136506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0aab4-2604-4276-8279-cfb01de77318 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:59.138385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 680/1364] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:59.144142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4091-7d81-471d-bab5-3b60348b075f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:59.145124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4091-7d81-471d-bab5-3b60348b075f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:59.145894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc4091-7d81-471d-bab5-3b60348b075f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:59.165162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5fcc4091-7d81-471d-bab5-3b60348b075f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:59.166354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5fcc4091-7d81-471d-bab5-3b60348b075f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:59.174703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 685/1365] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:47:59.221101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb91937-8192-46f3-8bd3-860725a71157 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:59.221399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb91937-8192-46f3-8bd3-860725a71157 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:59.221689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb91937-8192-46f3-8bd3-860725a71157 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:59.223110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 681/1366] 10.222.0.24 () {54 vars in 961 bytes} [Tue May 24 13:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:47:59.228263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e9051-0e74-4cf1-a0d2-e3e8677b17e5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:59.228557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e9051-0e74-4cf1-a0d2-e3e8677b17e5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:59.228841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e9051-0e74-4cf1-a0d2-e3e8677b17e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:59.246884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-670e9051-0e74-4cf1-a0d2-e3e8677b17e5 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:47:59.248095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-670e9051-0e74-4cf1-a0d2-e3e8677b17e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:47:59.258018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 686/1367] 10.222.0.24 () {58 vars in 1058 bytes} [Tue May 24 13:47:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:47:59.265957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb91937-8192-46f3-8bd3-860725a71157 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:47:59.281442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb91937-8192-46f3-8bd3-860725a71157 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:47:59.292352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:47:59.293030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:47:59.293317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:47:59.293595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:47:59.302495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:47:59.304129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99aa534-3519-4432-b9f2-74c89986ca9b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:47:59.316000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 682/1368] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:01.605279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b7b40-c9d0-4001-9f6f-25ec76502f9f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:01.605838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b7b40-c9d0-4001-9f6f-25ec76502f9f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:01.605838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b7b40-c9d0-4001-9f6f-25ec76502f9f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:01.607195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 687/1369] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:01.612749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0988ab-f109-44f6-baf4-26141d24d06d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:01.613217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0988ab-f109-44f6-baf4-26141d24d06d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:01.613324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0988ab-f109-44f6-baf4-26141d24d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:01.634124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2c0988ab-f109-44f6-baf4-26141d24d06d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:01.635374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2c0988ab-f109-44f6-baf4-26141d24d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:01.643552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 683/1370] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:01.688629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b7b40-c9d0-4001-9f6f-25ec76502f9f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:01.703977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b7b40-c9d0-4001-9f6f-25ec76502f9f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:01.710305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:01.711213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:01.711689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:01.712189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:01.721289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:01.723672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd8967-eac8-40f1-8ee0-5ea5ab7e1fd8 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:01.734638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 688/1371] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:01.823188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f2fc-bbb9-4764-b9b4-1f2d807a200a None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:01.823756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f2fc-bbb9-4764-b9b4-1f2d807a200a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:01.824231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f13f2fc-bbb9-4764-b9b4-1f2d807a200a None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:01.828469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 684/1372] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:01.831688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf48-0532-4a10-81e2-9066b97455fd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:01.832362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf48-0532-4a10-81e2-9066b97455fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:01.833603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4baf48-0532-4a10-81e2-9066b97455fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:01.853623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7a4baf48-0532-4a10-81e2-9066b97455fd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:01.855205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a4baf48-0532-4a10-81e2-9066b97455fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:01.864707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 689/1373] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:02.195246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd9e3-3229-4a92-966a-2f5ba5ebdeeb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:02.195754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd9e3-3229-4a92-966a-2f5ba5ebdeeb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:02.196231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd9e3-3229-4a92-966a-2f5ba5ebdeeb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:02.197766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 685/1374] 10.222.0.24 () {54 vars in 963 bytes} [Tue May 24 13:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:02.204060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30a34-30ec-4aa4-99bb-f4195528090d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:02.204445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30a34-30ec-4aa4-99bb-f4195528090d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:02.204795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f30a34-30ec-4aa4-99bb-f4195528090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:02.257487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-24f30a34-30ec-4aa4-99bb-f4195528090d None None] MFA Rules not processed for user `9da57903126d421b8ac6c0e26b3fb3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:02.258955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-24f30a34-30ec-4aa4-99bb-f4195528090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:02.330027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 690/1375] 10.222.0.24 () {58 vars in 1060 bytes} [Tue May 24 13:48:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:48:02.338131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efd9e3-3229-4a92-966a-2f5ba5ebdeeb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:02.349907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efd9e3-3229-4a92-966a-2f5ba5ebdeeb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:02.353495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:02.354164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:02.354456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:02.354729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:02.364522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:02.365953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e415ef9-badb-45ba-8ac9-e542e70f92ef service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:02.376686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 686/1376] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:48:04.012814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111040-a132-416e-a30c-d2bf107df4ea None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:04.013417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111040-a132-416e-a30c-d2bf107df4ea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:04.013417 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0111040-a132-416e-a30c-d2bf107df4ea None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:04.014816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 691/1377] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:04.021442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07c84-782f-482e-b697-82b48a62faf0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:04.021975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07c84-782f-482e-b697-82b48a62faf0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:04.022074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07c84-782f-482e-b697-82b48a62faf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:04.041449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6fb07c84-782f-482e-b697-82b48a62faf0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:04.042714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6fb07c84-782f-482e-b697-82b48a62faf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:04.051000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 687/1378] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:04.095785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0111040-a132-416e-a30c-d2bf107df4ea None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:04.111262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0111040-a132-416e-a30c-d2bf107df4ea None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:04.117185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:04.117859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:04.118150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:04.118431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:04.127966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:04.129847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc5845-96e0-4fa2-894a-4f526ace184c service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:04.134296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:04.141909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 692/1379] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:06.435952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d7ec-be1f-4374-b2ff-095617029544 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.436515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d7ec-be1f-4374-b2ff-095617029544 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.436515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d7ec-be1f-4374-b2ff-095617029544 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.437894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 688/1380] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:06.444229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8dca5-db09-4c6d-b2a4-7ab80809c314 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.444693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8dca5-db09-4c6d-b2a4-7ab80809c314 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.445208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8dca5-db09-4c6d-b2a4-7ab80809c314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.463383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a4b8dca5-db09-4c6d-b2a4-7ab80809c314 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:06.464747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4b8dca5-db09-4c6d-b2a4-7ab80809c314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:06.473064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 693/1381] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:06.520149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383d7ec-be1f-4374-b2ff-095617029544 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:06.534612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383d7ec-be1f-4374-b2ff-095617029544 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:06.540907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:06.541585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:06.541879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:06.542161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:06.550852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:06.553110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ac9ab-3c4f-48be-bc4e-242e8468e174 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:06.563303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 689/1382] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:08.821568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e2ad-7369-4194-b3a6-a863c83d17ee None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:08.822113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e2ad-7369-4194-b3a6-a863c83d17ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:08.822113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61e2ad-7369-4194-b3a6-a863c83d17ee None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:08.823501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 694/1383] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:08.829079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497647e-a9fe-4064-89a2-9f5910f12357 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:08.829508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497647e-a9fe-4064-89a2-9f5910f12357 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:08.829855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497647e-a9fe-4064-89a2-9f5910f12357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:08.849571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d497647e-a9fe-4064-89a2-9f5910f12357 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:08.850885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d497647e-a9fe-4064-89a2-9f5910f12357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:08.858859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 690/1384] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:08.903894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61e2ad-7369-4194-b3a6-a863c83d17ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:08.920351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61e2ad-7369-4194-b3a6-a863c83d17ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:08.925905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:08.927220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:08.927948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:08.928390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:08.937952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:08.939548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754401e1-d8d5-4ea4-867f-7360aa64d009 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:08.949412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 695/1385] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:11.388895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc723ac-0ca5-4098-a63f-5ea930b2f698 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:11.389421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc723ac-0ca5-4098-a63f-5ea930b2f698 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:11.389421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc723ac-0ca5-4098-a63f-5ea930b2f698 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:11.390805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 691/1386] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:11.396547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd062e4f-d89a-4ab2-9cec-5801b4427631 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:11.397049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd062e4f-d89a-4ab2-9cec-5801b4427631 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:11.397410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd062e4f-d89a-4ab2-9cec-5801b4427631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:11.420392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dd062e4f-d89a-4ab2-9cec-5801b4427631 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:11.422257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dd062e4f-d89a-4ab2-9cec-5801b4427631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:11.432990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 696/1387] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:11.476968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc723ac-0ca5-4098-a63f-5ea930b2f698 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:11.492183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc723ac-0ca5-4098-a63f-5ea930b2f698 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:11.497788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:11.498452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:11.498744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:11.499025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:11.509031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:11.511051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980dfb4-fdae-4da8-afaa-3748624c27b1 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:11.524349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 692/1388] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:14.198484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8d80-c5a0-46d8-be21-55f6c5d87a50 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:14.198999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8d80-c5a0-46d8-be21-55f6c5d87a50 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:14.198999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8d80-c5a0-46d8-be21-55f6c5d87a50 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:14.200379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 697/1389] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:14.205661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da934-ccaf-4d86-86e0-1ee19783600e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:14.206248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da934-ccaf-4d86-86e0-1ee19783600e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:14.206248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149da934-ccaf-4d86-86e0-1ee19783600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:14.224542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-149da934-ccaf-4d86-86e0-1ee19783600e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:14.225675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-149da934-ccaf-4d86-86e0-1ee19783600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:14.233656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 693/1390] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:48:14.279226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8d80-c5a0-46d8-be21-55f6c5d87a50 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:14.296094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8d80-c5a0-46d8-be21-55f6c5d87a50 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:14.303607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de99254d-4138-424e-a424-894622aa6943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:14.304771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99254d-4138-424e-a424-894622aa6943 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:14.305144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99254d-4138-424e-a424-894622aa6943 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:14.305504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de99254d-4138-424e-a424-894622aa6943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:14.316381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99254d-4138-424e-a424-894622aa6943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:14.318465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de99254d-4138-424e-a424-894622aa6943 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:14.330846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 698/1391] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:16.909492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e689-6a81-4a36-922e-cba80f59bf1b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:16.910104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e689-6a81-4a36-922e-cba80f59bf1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:16.910104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e689-6a81-4a36-922e-cba80f59bf1b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:16.911799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 694/1392] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:16.917021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3b98-9bb4-4c53-b341-1253342835fd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:16.917477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3b98-9bb4-4c53-b341-1253342835fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:16.917588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3b98-9bb4-4c53-b341-1253342835fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:16.936262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8b9e3b98-9bb4-4c53-b341-1253342835fd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:16.937462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8b9e3b98-9bb4-4c53-b341-1253342835fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:16.946227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 699/1393] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:16.990624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e689-6a81-4a36-922e-cba80f59bf1b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:17.005455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e689-6a81-4a36-922e-cba80f59bf1b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:17.010949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:17.011584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:17.011907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:17.012193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:17.021238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:17.022869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4a141-730f-436f-bad3-9b7db6784ddd service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:17.033668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 695/1394] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:18.125519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7b02-bdc5-43b1-8e92-afafd16181fb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.126107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7b02-bdc5-43b1-8e92-afafd16181fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.126107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7b02-bdc5-43b1-8e92-afafd16181fb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.127450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 700/1395] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:18.132978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270768e-bf2a-48e9-a92d-b29454ffe0d6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.133417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270768e-bf2a-48e9-a92d-b29454ffe0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.133579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e270768e-bf2a-48e9-a92d-b29454ffe0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.139801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=52 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:18.153108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e270768e-bf2a-48e9-a92d-b29454ffe0d6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:18.154328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e270768e-bf2a-48e9-a92d-b29454ffe0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:18.162946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 696/1396] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:18.217595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470cc6-2fa1-4a5e-9fa0-22fc53e8a92b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.217966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470cc6-2fa1-4a5e-9fa0-22fc53e8a92b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.218329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d470cc6-2fa1-4a5e-9fa0-22fc53e8a92b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.219756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 701/1397] 10.222.0.24 () {54 vars in 962 bytes} [Tue May 24 13:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:18.225490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788956f-483f-4a84-b239-3e99092eedfd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.225789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788956f-483f-4a84-b239-3e99092eedfd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.226103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788956f-483f-4a84-b239-3e99092eedfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.275294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b788956f-483f-4a84-b239-3e99092eedfd None None] MFA Rules not processed for user `eb86cbb65b734fc69bf7604414f8c9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:18.276616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b788956f-483f-4a84-b239-3e99092eedfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:18.336831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 697/1398] 10.222.0.24 () {58 vars in 1059 bytes} [Tue May 24 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:18.344533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470cc6-2fa1-4a5e-9fa0-22fc53e8a92b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:18.356000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d470cc6-2fa1-4a5e-9fa0-22fc53e8a92b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:18.359538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:18.360220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.360512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.360795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.369800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:18.371243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022f10e-49ef-4236-a145-0e42128c64e1 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:18.381194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 702/1399] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:18.945721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250f1e-4f2d-4c1f-b3c9-eb158bca8573 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:18.946016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250f1e-4f2d-4c1f-b3c9-eb158bca8573 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:18.946297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4250f1e-4f2d-4c1f-b3c9-eb158bca8573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:18.998268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d4250f1e-4f2d-4c1f-b3c9-eb158bca8573 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:18.999473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d4250f1e-4f2d-4c1f-b3c9-eb158bca8573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:19.070377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 698/1400] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:48:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:19.114411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e035b2-3a10-4c52-a121-c24d44c39265 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.114972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e035b2-3a10-4c52-a121-c24d44c39265 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.115426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e035b2-3a10-4c52-a121-c24d44c39265 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.117005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 703/1401] 10.222.0.24 () {54 vars in 942 bytes} [Tue May 24 13:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:19.124623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08543a-c737-4a21-9f4d-1c47a10dc62c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.124918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08543a-c737-4a21-9f4d-1c47a10dc62c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.125442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08543a-c737-4a21-9f4d-1c47a10dc62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.174506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bf08543a-c737-4a21-9f4d-1c47a10dc62c None None] MFA Rules not processed for user `f0bf046435a84f7592fd022c03eda0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:19.175718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bf08543a-c737-4a21-9f4d-1c47a10dc62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:19.188939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:19.247908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 699/1402] 10.222.0.24 () {58 vars in 1039 bytes} [Tue May 24 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:19.257901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e035b2-3a10-4c52-a121-c24d44c39265 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:19.273026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e035b2-3a10-4c52-a121-c24d44c39265 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:19.278078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:19.279000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.279292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.279568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.286619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ca56-732e-4414-85fa-19fecdbcaaf4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.287199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ca56-732e-4414-85fa-19fecdbcaaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.287483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d31ca56-732e-4414-85fa-19fecdbcaaf4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.288513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:19.289044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 700/1403] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:48:19.290336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:19.296761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24669-50d8-42ec-ae8f-2c05b384091d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.297229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24669-50d8-42ec-ae8f-2c05b384091d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.297590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c24669-50d8-42ec-ae8f-2c05b384091d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.299030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 704/1404] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:48:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:48:19.319117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-51c24669-50d8-42ec-ae8f-2c05b384091d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:19.320618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51c24669-50d8-42ec-ae8f-2c05b384091d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:19.332611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 701/1405] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:19.378855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:19.392214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6ee2b-3673-4221-b038-f06008c1d1bc service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:19.398439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:19.398987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:19.399279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:19.399562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:19.413454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:19.415073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b607855-68d9-475b-8f5a-b29591d1806d service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:19.430705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 705/1406] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:21.719062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05521d-a63d-427d-bf10-c9518bbc4f10 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:21.719598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05521d-a63d-427d-bf10-c9518bbc4f10 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:21.719715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05521d-a63d-427d-bf10-c9518bbc4f10 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:21.721198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 702/1407] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:21.727628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986949-6cbf-407f-997d-5d1583fa2ed3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:21.728145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986949-6cbf-407f-997d-5d1583fa2ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:21.728465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65986949-6cbf-407f-997d-5d1583fa2ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:21.749641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65986949-6cbf-407f-997d-5d1583fa2ed3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:21.751093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65986949-6cbf-407f-997d-5d1583fa2ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:21.758998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 706/1408] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:48:21.804993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05521d-a63d-427d-bf10-c9518bbc4f10 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:21.820334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05521d-a63d-427d-bf10-c9518bbc4f10 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:21.826000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:21.826724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:21.827087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:21.827377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:21.837275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:21.838848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea731ae-7f87-45d8-8ab7-93b9795f3b3f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:21.851609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 703/1409] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:31.744190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=43 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:32.273738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=33 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:48:43.038832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd59f-64cb-4990-902c-51fedecf2b12 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:43.039444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd59f-64cb-4990-902c-51fedecf2b12 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:43.039444 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317cd59f-64cb-4990-902c-51fedecf2b12 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:43.041816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 707/1410] 10.222.0.24 () {54 vars in 918 bytes} [Tue May 24 13:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:43.046878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00747-2768-4417-85a4-1588f61386d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:43.047283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00747-2768-4417-85a4-1588f61386d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:43.047449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e00747-2768-4417-85a4-1588f61386d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:43.065168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-75e00747-2768-4417-85a4-1588f61386d1 None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:43.066386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-75e00747-2768-4417-85a4-1588f61386d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:43.074919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 704/1411] 10.222.0.24 () {58 vars in 1015 bytes} [Tue May 24 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:43.089590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac31b-c4bb-420c-8194-5eed9525a762 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:43.089881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac31b-c4bb-420c-8194-5eed9525a762 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:43.090163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dac31b-c4bb-420c-8194-5eed9525a762 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:43.091524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 708/1412] 10.222.0.24 () {54 vars in 918 bytes} [Tue May 24 13:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:43.095941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383eced-cac6-4fd6-a95f-a994ed3c96ae None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:43.096240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383eced-cac6-4fd6-a95f-a994ed3c96ae None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:43.096525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383eced-cac6-4fd6-a95f-a994ed3c96ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:43.114562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6383eced-cac6-4fd6-a95f-a994ed3c96ae None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:43.115750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6383eced-cac6-4fd6-a95f-a994ed3c96ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:43.124908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 705/1413] 10.222.0.24 () {58 vars in 1015 bytes} [Tue May 24 13:48:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:45.805781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53a8fc-eeab-47b5-9383-63cd43581c45 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:45.806317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53a8fc-eeab-47b5-9383-63cd43581c45 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:45.806317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc53a8fc-eeab-47b5-9383-63cd43581c45 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:45.807604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:48:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:45.812908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a4b3-6491-43be-9ec9-85d0180779c5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:45.813289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a4b3-6491-43be-9ec9-85d0180779c5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:45.813477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a4b3-6491-43be-9ec9-85d0180779c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:45.834891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5651a4b3-6491-43be-9ec9-85d0180779c5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:45.836368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5651a4b3-6491-43be-9ec9-85d0180779c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:45.844158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:45.887101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53a8fc-eeab-47b5-9383-63cd43581c45 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:45.902078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc53a8fc-eeab-47b5-9383-63cd43581c45 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:45.907577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:45.908272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:45.908565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:45.908848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:45.917275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:45.918859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f562aa-4012-4769-acb3-ed0dad6400eb service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:45.928118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 710/1416] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:46.897006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b74c-9c9a-4acd-a021-eb662fa81c95 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:46.897530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b74c-9c9a-4acd-a021-eb662fa81c95 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:46.897530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346b74c-9c9a-4acd-a021-eb662fa81c95 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:46.899771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:46.904648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4a7-4cfb-4f87-a111-cc6276db8801 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:46.905061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4a7-4cfb-4f87-a111-cc6276db8801 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:46.905226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59f4a7-4cfb-4f87-a111-cc6276db8801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:46.923806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4e59f4a7-4cfb-4f87-a111-cc6276db8801 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:46.925033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e59f4a7-4cfb-4f87-a111-cc6276db8801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:46.932848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:47.083675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346b74c-9c9a-4acd-a021-eb662fa81c95 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:47.098454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346b74c-9c9a-4acd-a021-eb662fa81c95 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:47.103846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:47.104503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:47.104795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:47.105081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:47.113102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:47.114675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9761e-5760-44cd-ab60-b6a6f1f0d90a service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:47.124001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 708/1419] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:48.047175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc5e-76e9-406e-82f6-5ed5f0d15ee9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:48.047726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc5e-76e9-406e-82f6-5ed5f0d15ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:48.047726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129dc5e-76e9-406e-82f6-5ed5f0d15ee9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:48.049152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:48.054719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58db3-12dc-4a3e-9567-9556e4d0f5da None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:48.055147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58db3-12dc-4a3e-9567-9556e4d0f5da None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:48.055335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58db3-12dc-4a3e-9567-9556e4d0f5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:48.073585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-66c58db3-12dc-4a3e-9567-9556e4d0f5da None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:48.074780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66c58db3-12dc-4a3e-9567-9556e4d0f5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:48.082490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:48.231627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129dc5e-76e9-406e-82f6-5ed5f0d15ee9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:48.246438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129dc5e-76e9-406e-82f6-5ed5f0d15ee9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:48.251683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:48.252603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:48.252902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:48.253170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:48.261454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:48.263026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227256e2-aeec-42de-8a63-43414b13e351 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:48.272213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 713/1422] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:49.175340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71779e-c846-455e-ac65-5eb4f3c4567c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:49.175881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71779e-c846-455e-ac65-5eb4f3c4567c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:49.176015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71779e-c846-455e-ac65-5eb4f3c4567c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:49.177385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:49.182967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396877eb-c998-4585-9059-ecfc73606899 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:49.183393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396877eb-c998-4585-9059-ecfc73606899 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:49.183534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396877eb-c998-4585-9059-ecfc73606899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:49.201723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-396877eb-c998-4585-9059-ecfc73606899 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:49.202932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-396877eb-c998-4585-9059-ecfc73606899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:49.210768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:49.360397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71779e-c846-455e-ac65-5eb4f3c4567c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:49.375402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71779e-c846-455e-ac65-5eb4f3c4567c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:49.381016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:49.381652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:49.381942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:49.382493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:49.390930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:49.392541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d397b-3553-4b29-a0ab-cd6b5349f305 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:49.402017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 711/1425] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:50.327494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15b5f-d6bf-4ee2-8325-62367bf546bb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:50.328095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15b5f-d6bf-4ee2-8325-62367bf546bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:50.328095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef15b5f-d6bf-4ee2-8325-62367bf546bb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:50.329506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:50.334330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de79b4-d06d-4fbc-b512-b8d284f5f05e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:50.334735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de79b4-d06d-4fbc-b512-b8d284f5f05e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:50.334899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de79b4-d06d-4fbc-b512-b8d284f5f05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:50.354162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b2de79b4-d06d-4fbc-b512-b8d284f5f05e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:50.355374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b2de79b4-d06d-4fbc-b512-b8d284f5f05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:50.363111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:50.512698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef15b5f-d6bf-4ee2-8325-62367bf546bb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:50.527341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef15b5f-d6bf-4ee2-8325-62367bf546bb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:50.532799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:50.533430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:50.533724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:50.534008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:50.542705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:50.544328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df0417-ebcb-48b7-abcf-c1443ac5b98d service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:50.553575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 716/1428] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:51.408719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3609e-4ef2-4d24-8db8-0dc9352c81a8 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:51.409197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3609e-4ef2-4d24-8db8-0dc9352c81a8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:51.409312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3609e-4ef2-4d24-8db8-0dc9352c81a8 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:51.410704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:51.416234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab9f0-37af-49a8-8bb2-8cee532ccaff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:51.416646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab9f0-37af-49a8-8bb2-8cee532ccaff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:51.416799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ab9f0-37af-49a8-8bb2-8cee532ccaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:51.435155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b89ab9f0-37af-49a8-8bb2-8cee532ccaff None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:51.436394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b89ab9f0-37af-49a8-8bb2-8cee532ccaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:51.444086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:51.595596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3609e-4ef2-4d24-8db8-0dc9352c81a8 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:51.610227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3609e-4ef2-4d24-8db8-0dc9352c81a8 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:51.615456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:51.616124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:51.616416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:51.617242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:51.625263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:51.626781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae58f1d-18d8-4393-b442-471f114aa60f service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:51.636279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 714/1431] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:52.505311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b10d9-6b20-4aa1-addc-f15b80c7c574 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:52.505858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b10d9-6b20-4aa1-addc-f15b80c7c574 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:52.505999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893b10d9-6b20-4aa1-addc-f15b80c7c574 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:52.507957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:52.513259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8760-c6c5-47d5-a405-74a6df0edc00 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:52.513716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8760-c6c5-47d5-a405-74a6df0edc00 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:52.513816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c8760-c6c5-47d5-a405-74a6df0edc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:52.532225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5c7c8760-c6c5-47d5-a405-74a6df0edc00 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:52.533421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5c7c8760-c6c5-47d5-a405-74a6df0edc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:52.541123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:52.689986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b10d9-6b20-4aa1-addc-f15b80c7c574 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:52.704533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893b10d9-6b20-4aa1-addc-f15b80c7c574 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:52.709781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:52.710595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:52.710881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:52.711160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:52.718981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:52.720613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b90794-510b-4863-a13e-99ac0c4076c2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:52.729891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 719/1434] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:53.579101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0742e9a-4012-4965-b23a-47330825cf3e None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:53.579672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0742e9a-4012-4965-b23a-47330825cf3e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:53.579672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0742e9a-4012-4965-b23a-47330825cf3e None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:53.581098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:53.586454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa20ba-431b-4619-88fe-b2fd791cd9d3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:53.586862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa20ba-431b-4619-88fe-b2fd791cd9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:53.587048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa20ba-431b-4619-88fe-b2fd791cd9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:53.605093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b3fa20ba-431b-4619-88fe-b2fd791cd9d3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:53.606293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b3fa20ba-431b-4619-88fe-b2fd791cd9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:53.614249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:53.772080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0742e9a-4012-4965-b23a-47330825cf3e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:53.787470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0742e9a-4012-4965-b23a-47330825cf3e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:53.793476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:53.794125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:53.794412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:53.794702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:53.802892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:53.805259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625e069-5620-4688-8a05-472a83f06bba service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:53.814627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 717/1437] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:54.667279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c00f6-31e8-4d03-9f23-85903dab51fb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:54.667879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c00f6-31e8-4d03-9f23-85903dab51fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:54.667879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c00f6-31e8-4d03-9f23-85903dab51fb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:54.669269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:54.674074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c8e88-e373-4bd1-898c-1eee67a9f8f7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:54.674480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c8e88-e373-4bd1-898c-1eee67a9f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:54.674646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c8e88-e373-4bd1-898c-1eee67a9f8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:54.693083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7b7c8e88-e373-4bd1-898c-1eee67a9f8f7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:54.694285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7b7c8e88-e373-4bd1-898c-1eee67a9f8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:54.702311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:54.849590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c00f6-31e8-4d03-9f23-85903dab51fb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:54.863735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c00f6-31e8-4d03-9f23-85903dab51fb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:54.869609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:54.870271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:54.870564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:54.870847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:54.879137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:54.880767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afefeae-3edc-4ffa-ac8a-4a737c27932c service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:54.890688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 722/1440] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:56.982598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:56.983188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:56.983188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:56.984513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:56.989883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d6-ee61-4624-8e28-431288be3602 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:56.990330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d6-ee61-4624-8e28-431288be3602 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:56.990483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7268d6-ee61-4624-8e28-431288be3602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:57.008780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0f7268d6-ee61-4624-8e28-431288be3602 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:57.009973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0f7268d6-ee61-4624-8e28-431288be3602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:57.017686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:57.056531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:57.058206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:57.071670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3286177-6186-45b6-8eaf-845960b7e760 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:57.075198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:57.075873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:57.076167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:57.076448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:57.085895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:57.087632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7382e1-d0c7-43a8-979c-2e15c6e5687a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:57.102600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 720/1443] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:48:59.472263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7f1c5-0aac-4f1c-98aa-bffc92f99ef7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:59.472787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7f1c5-0aac-4f1c-98aa-bffc92f99ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:59.472787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf7f1c5-0aac-4f1c-98aa-bffc92f99ef7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:59.474091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:48:59.480281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8b8da-69cf-4cee-9590-7028fd3f3506 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:59.480797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8b8da-69cf-4cee-9590-7028fd3f3506 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:59.480907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8b8da-69cf-4cee-9590-7028fd3f3506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:59.499536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-96f8b8da-69cf-4cee-9590-7028fd3f3506 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:48:59.500766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-96f8b8da-69cf-4cee-9590-7028fd3f3506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:48:59.509180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:48:59.547543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7f1c5-0aac-4f1c-98aa-bffc92f99ef7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:48:59.558259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf7f1c5-0aac-4f1c-98aa-bffc92f99ef7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:48:59.561772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:48:59.562407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:48:59.562692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:48:59.562970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:48:59.571834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:48:59.573276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e31c63-5b3e-4bad-b163-d33b43d9ec11 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:48:59.582426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 725/1446] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:02.167810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929194-c6cc-4a7f-98a1-27df949e8935 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:02.168367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929194-c6cc-4a7f-98a1-27df949e8935 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:02.168463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd929194-c6cc-4a7f-98a1-27df949e8935 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:02.169855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:02.175247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b2bb3-a5fa-4758-ae65-b1b46deed7bf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:02.175672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b2bb3-a5fa-4758-ae65-b1b46deed7bf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:02.175842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b2bb3-a5fa-4758-ae65-b1b46deed7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:02.194217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a86b2bb3-a5fa-4758-ae65-b1b46deed7bf None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:02.195728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a86b2bb3-a5fa-4758-ae65-b1b46deed7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:02.203536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:49:02.242814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929194-c6cc-4a7f-98a1-27df949e8935 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:02.253623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd929194-c6cc-4a7f-98a1-27df949e8935 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:02.257102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:02.258005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:02.258415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:02.258712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:02.267467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:02.268984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b7126-05a2-4c37-8611-452f4c46c116 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:02.279629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 723/1449] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:04.610917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610715-ec53-41b4-b555-d0c7ae019c88 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:04.611498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610715-ec53-41b4-b555-d0c7ae019c88 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:04.611498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0610715-ec53-41b4-b555-d0c7ae019c88 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:04.612938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:04.618953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d25cb-8b9e-4871-bad1-b1b4508bb060 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:04.619398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d25cb-8b9e-4871-bad1-b1b4508bb060 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:04.619502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d25cb-8b9e-4871-bad1-b1b4508bb060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:04.638225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f37d25cb-8b9e-4871-bad1-b1b4508bb060 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:04.639440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f37d25cb-8b9e-4871-bad1-b1b4508bb060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:04.647134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:49:04.685121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0610715-ec53-41b4-b555-d0c7ae019c88 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:04.696501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0610715-ec53-41b4-b555-d0c7ae019c88 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:04.699956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:04.700599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:04.700884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:04.701162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:04.710260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:04.711897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829324b-2876-4377-8a6c-47b6c1f431c6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:04.721127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 728/1452] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:07.005932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7115-67fb-49bb-852e-fb973bb9e1e1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:07.006489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7115-67fb-49bb-852e-fb973bb9e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:07.006489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe7115-67fb-49bb-852e-fb973bb9e1e1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:07.008278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:07.013765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c16802-a62d-4dda-bbbd-54744d618b15 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:07.014192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c16802-a62d-4dda-bbbd-54744d618b15 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:07.014338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c16802-a62d-4dda-bbbd-54744d618b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:07.032459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a8c16802-a62d-4dda-bbbd-54744d618b15 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:07.033663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8c16802-a62d-4dda-bbbd-54744d618b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:07.041983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:07.078535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7115-67fb-49bb-852e-fb973bb9e1e1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:07.089708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe7115-67fb-49bb-852e-fb973bb9e1e1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:07.093165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:07.093794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:07.094079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:07.094362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:07.104241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:07.105667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc758e90-7d3e-4cbe-b631-dbf8c640079c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:07.115036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 726/1455] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:09.376281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae35d2-8349-4ea0-a4ac-04c61f7c7403 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:09.376805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae35d2-8349-4ea0-a4ac-04c61f7c7403 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:09.376805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae35d2-8349-4ea0-a4ac-04c61f7c7403 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:09.378382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:09.384634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba44f-d542-41b2-9390-1dd03de037ba None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:09.385105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba44f-d542-41b2-9390-1dd03de037ba None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:09.385216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ba44f-d542-41b2-9390-1dd03de037ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:09.402735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d61ba44f-d542-41b2-9390-1dd03de037ba None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:09.403968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d61ba44f-d542-41b2-9390-1dd03de037ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:09.411590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:09.450323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae35d2-8349-4ea0-a4ac-04c61f7c7403 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:09.460980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae35d2-8349-4ea0-a4ac-04c61f7c7403 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:09.464645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:09.465272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:09.465563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:09.465841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:09.474579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:09.476563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f346ec-cd9f-4076-8bb0-30e069fb564d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:09.486048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 731/1458] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:11.743569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144333c0-1717-499d-954d-9f73ceba701f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:11.744162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144333c0-1717-499d-954d-9f73ceba701f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:11.744268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144333c0-1717-499d-954d-9f73ceba701f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:11.745582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:11.751070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175c0e-711b-4fc2-a2be-021338ec55b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:11.751461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175c0e-711b-4fc2-a2be-021338ec55b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:11.751635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175c0e-711b-4fc2-a2be-021338ec55b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:11.758523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=70 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:11.771624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-47175c0e-711b-4fc2-a2be-021338ec55b2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:11.772841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-47175c0e-711b-4fc2-a2be-021338ec55b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:11.781091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:11.817645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144333c0-1717-499d-954d-9f73ceba701f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:11.828825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144333c0-1717-499d-954d-9f73ceba701f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:11.832312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:11.832949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:11.833237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:11.833517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:11.842851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:11.844323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d8986-b214-4b8e-a286-c9940f3266db service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:11.853578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 729/1461] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:14.111759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11dcb-fd8f-4910-b369-8f74f9dab294 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:14.112340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11dcb-fd8f-4910-b369-8f74f9dab294 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:14.112340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a11dcb-fd8f-4910-b369-8f74f9dab294 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:14.113749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:14.119433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d636-64b4-4970-b604-9719ab6431af None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:14.119856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d636-64b4-4970-b604-9719ab6431af None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:14.120039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11d636-64b4-4970-b604-9719ab6431af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:14.125994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:14.139475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a11d636-64b4-4970-b604-9719ab6431af None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:14.140686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a11d636-64b4-4970-b604-9719ab6431af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:14.149201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:14.185319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a11dcb-fd8f-4910-b369-8f74f9dab294 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:14.196594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a11dcb-fd8f-4910-b369-8f74f9dab294 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:14.200037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:14.200663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:14.200945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:14.201252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:14.209751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:14.212121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f708403-4036-4efc-b7d3-c2a40c9057f3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:14.221300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 734/1464] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:16.434409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986f1d-1c83-421a-9109-18ac84a8f495 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:16.434990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986f1d-1c83-421a-9109-18ac84a8f495 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:16.434990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986f1d-1c83-421a-9109-18ac84a8f495 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:16.436379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:16.441522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ff1d-cbfa-4245-9ff9-5f69fca5a14a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:16.441946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ff1d-cbfa-4245-9ff9-5f69fca5a14a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:16.442090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03ff1d-cbfa-4245-9ff9-5f69fca5a14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:16.459337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0b03ff1d-cbfa-4245-9ff9-5f69fca5a14a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:16.460633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b03ff1d-cbfa-4245-9ff9-5f69fca5a14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:16.468228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:16.505705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93986f1d-1c83-421a-9109-18ac84a8f495 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:16.516532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93986f1d-1c83-421a-9109-18ac84a8f495 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:16.520039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:16.520678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:16.520967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:16.521248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:16.530466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:16.531931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f00d2-8679-4a85-a2f4-dd8e893622e3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:16.541938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 732/1467] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:18.742443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784a70-5778-4356-b50a-86418e5edcdc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:18.742972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784a70-5778-4356-b50a-86418e5edcdc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:18.743090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4784a70-5778-4356-b50a-86418e5edcdc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:18.744430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:49:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:18.749741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b68ed-636a-4940-a8a4-72909f8d34fc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:18.750139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b68ed-636a-4940-a8a4-72909f8d34fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:18.750314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b68ed-636a-4940-a8a4-72909f8d34fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:18.768028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-919b68ed-636a-4940-a8a4-72909f8d34fc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:18.769167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-919b68ed-636a-4940-a8a4-72909f8d34fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:18.777408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:18.813410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4784a70-5778-4356-b50a-86418e5edcdc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:18.823921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4784a70-5778-4356-b50a-86418e5edcdc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:18.827396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:18.828065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:18.828354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:18.828634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:18.837942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:18.839414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859b192-124f-4783-a0b4-4470a9dfe002 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:18.848641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 737/1470] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:28.762873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=11 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:28.844547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=17 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:49:52.226445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb75f0-6ec9-4ba9-a911-9c41c94dde85 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:52.227106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb75f0-6ec9-4ba9-a911-9c41c94dde85 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:52.227106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb75f0-6ec9-4ba9-a911-9c41c94dde85 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:52.228518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 734/1471] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:52.233532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542bf33-5a71-4d8a-a19e-9b7937052837 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:52.233982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542bf33-5a71-4d8a-a19e-9b7937052837 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:52.234095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542bf33-5a71-4d8a-a19e-9b7937052837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:52.251613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d542bf33-5a71-4d8a-a19e-9b7937052837 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:52.252843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d542bf33-5a71-4d8a-a19e-9b7937052837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:52.261333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 738/1472] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:52.308133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13c00-0ca4-4825-8084-e21a2529aa82 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:52.308421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13c00-0ca4-4825-8084-e21a2529aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:52.308699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d13c00-0ca4-4825-8084-e21a2529aa82 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:52.310056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 735/1473] 10.222.0.24 () {54 vars in 962 bytes} [Tue May 24 13:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:52.314685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7168-545e-4254-afc9-96b88ca895c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:52.314981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7168-545e-4254-afc9-96b88ca895c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:52.315293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d7168-545e-4254-afc9-96b88ca895c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:52.333325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-223d7168-545e-4254-afc9-96b88ca895c0 None None] MFA Rules not processed for user `eb86cbb65b734fc69bf7604414f8c9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:52.334490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-223d7168-545e-4254-afc9-96b88ca895c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:52.341537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 739/1474] 10.222.0.24 () {58 vars in 1059 bytes} [Tue May 24 13:49:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:52.348765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13c00-0ca4-4825-8084-e21a2529aa82 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:52.359390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d13c00-0ca4-4825-8084-e21a2529aa82 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:52.362866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:52.363496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:52.363788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:52.364104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:52.373197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:52.374627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2219bee-f6c1-49ff-aee1-067562ddde60 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:52.384838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 736/1475] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:54.680155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4500d-6533-48be-a68f-79e1b96eba09 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:54.680716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4500d-6533-48be-a68f-79e1b96eba09 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:54.680716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4500d-6533-48be-a68f-79e1b96eba09 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:54.682061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 740/1476] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:54.687300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f367d4-ed43-4870-b616-df754064d5f1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:54.687695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f367d4-ed43-4870-b616-df754064d5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:54.687917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f367d4-ed43-4870-b616-df754064d5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:54.705870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-45f367d4-ed43-4870-b616-df754064d5f1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:54.707063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-45f367d4-ed43-4870-b616-df754064d5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:54.715358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 737/1477] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:49:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:54.737841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4500d-6533-48be-a68f-79e1b96eba09 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:54.750130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4500d-6533-48be-a68f-79e1b96eba09 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:54.754820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:54.755462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:54.755749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:54.756059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:54.756812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:54.759405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:49:54.759405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:49:54.759405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:49:54.760527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdc924-1279-4a0b-992a-3f855257529f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:54.779437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 741/1478] 10.222.0.24 () {56 vars in 1108 bytes} [Tue May 24 13:49:54 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:49:57.039557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d7f2-cb42-4c17-a42a-8708661a93eb None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:57.040149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d7f2-cb42-4c17-a42a-8708661a93eb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:57.040149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d7f2-cb42-4c17-a42a-8708661a93eb None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:57.041550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 738/1479] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:57.046800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48e29d-3c41-4d7d-a108-a15659c3963a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:57.047191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48e29d-3c41-4d7d-a108-a15659c3963a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:57.047364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e48e29d-3c41-4d7d-a108-a15659c3963a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:57.065599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4e48e29d-3c41-4d7d-a108-a15659c3963a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:57.066795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e48e29d-3c41-4d7d-a108-a15659c3963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:57.075418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 742/1480] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:57.118886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6d7f2-cb42-4c17-a42a-8708661a93eb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:57.133824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda6d7f2-cb42-4c17-a42a-8708661a93eb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:57.139115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:57.139751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:57.140067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:57.140396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:57.148486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:57.150069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609eb0-9b20-42c8-98c2-ad6776f41ae6 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:57.159275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 739/1481] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:49:59.575327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439817ee-74f3-467a-a9ef-e1578e0e01d7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:59.575858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439817ee-74f3-467a-a9ef-e1578e0e01d7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:59.575984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439817ee-74f3-467a-a9ef-e1578e0e01d7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:59.577268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 743/1482] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:49:59.582620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2049977-7b00-40d1-922c-7acb98361049 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:59.583018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2049977-7b00-40d1-922c-7acb98361049 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:59.583190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2049977-7b00-40d1-922c-7acb98361049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:59.601713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d2049977-7b00-40d1-922c-7acb98361049 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:49:59.602914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d2049977-7b00-40d1-922c-7acb98361049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:49:59.610603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 740/1483] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:49:59.645671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439817ee-74f3-467a-a9ef-e1578e0e01d7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:49:59.656891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439817ee-74f3-467a-a9ef-e1578e0e01d7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:49:59.660416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:49:59.661053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:49:59.661345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:49:59.661625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:49:59.672807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:49:59.674263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b40da-90dc-4694-b456-367c1d1dbc7a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:49:59.684843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 744/1484] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:50:01.969664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0517deb-e4ff-44d6-82d2-9f6e50b9f567 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:01.970198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0517deb-e4ff-44d6-82d2-9f6e50b9f567 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:01.970198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0517deb-e4ff-44d6-82d2-9f6e50b9f567 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:01.971554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 741/1485] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:50:01.977413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ddec-d7ef-494c-bd76-2d31007e8ee9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:01.977800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ddec-d7ef-494c-bd76-2d31007e8ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:01.977987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430ddec-d7ef-494c-bd76-2d31007e8ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:01.996112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3430ddec-d7ef-494c-bd76-2d31007e8ee9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:50:01.997315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3430ddec-d7ef-494c-bd76-2d31007e8ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:50:02.005489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 745/1486] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:50:02.043163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0517deb-e4ff-44d6-82d2-9f6e50b9f567 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:50:02.053847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0517deb-e4ff-44d6-82d2-9f6e50b9f567 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:50:02.057976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:50:02.058623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:02.058915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:02.059220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:02.068141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:50:02.069620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1910e-7854-4d40-bcb7-73ba4beef46a service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:50:02.079302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 742/1487] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:50:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:50:04.379921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e23e-ac1f-4c14-b313-454ff642eea4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:04.380583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e23e-ac1f-4c14-b313-454ff642eea4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:04.380583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e23e-ac1f-4c14-b313-454ff642eea4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:04.381842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 746/1488] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:50:04.387490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d3d7-65a7-4136-b76b-9d1bd52f2ee2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:04.387948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d3d7-65a7-4136-b76b-9d1bd52f2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:04.388140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93d3d7-65a7-4136-b76b-9d1bd52f2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:04.406249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6e93d3d7-65a7-4136-b76b-9d1bd52f2ee2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:50:04.407439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6e93d3d7-65a7-4136-b76b-9d1bd52f2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:50:04.415545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 743/1489] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:50:04.455984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4e23e-ac1f-4c14-b313-454ff642eea4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:50:04.466750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4e23e-ac1f-4c14-b313-454ff642eea4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:50:04.470258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:50:04.470895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:04.471179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:04.471467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:04.480525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:50:04.481954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236f06b-9cc1-41ec-9ce9-e58eb2fcbbd1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:50:04.491315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 747/1490] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:50:06.885296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a46a-c11c-49fa-b2b8-d3ba95efeacf None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:06.885861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a46a-c11c-49fa-b2b8-d3ba95efeacf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:06.885961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a46a-c11c-49fa-b2b8-d3ba95efeacf None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:06.887275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 744/1491] 10.222.0.24 () {54 vars in 926 bytes} [Tue May 24 13:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:50:06.892414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8267a9-83de-4fbf-bda1-6bbd36b73dfb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:06.892817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8267a9-83de-4fbf-bda1-6bbd36b73dfb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:06.892972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8267a9-83de-4fbf-bda1-6bbd36b73dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:06.910364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6b8267a9-83de-4fbf-bda1-6bbd36b73dfb None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:50:06.911645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6b8267a9-83de-4fbf-bda1-6bbd36b73dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:50:06.919909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 748/1492] 10.222.0.24 () {58 vars in 1023 bytes} [Tue May 24 13:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:50:06.957834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433a46a-c11c-49fa-b2b8-d3ba95efeacf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:50:06.968580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433a46a-c11c-49fa-b2b8-d3ba95efeacf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:50:06.971987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:50:06.972614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:50:06.972902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:50:06.973179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:50:06.982781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:50:06.984230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07895d42-d9a7-4bcd-a7ce-4d7e08cf5448 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:50:06.994259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 745/1493] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:50:16.905424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=33 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:50:16.989260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=33 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:51:47.740649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:47.741237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:47.741237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:47.759578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:51:47.760736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:47.768410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 749/1494] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:51:47.777535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:47.778032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:47.778146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:47.796929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:51:47.798153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:47.808369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 746/1495] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:51:47.812876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:47.822625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ccc662-c7a8-4627-ac34-8786b31ef1aa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:47.827098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:47.827760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:47.828088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:47.828375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:47.829117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:51:47.829430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:47.830468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:47.830468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:51:47.830468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:51:47.831348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:47.842876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 750/1496] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:51:47.851683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:47.860712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07cb0c-d0a5-49cc-b2a4-04471c1c1a11 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:47.865884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:47.866537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:47.866825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:47.867106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:47.867799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:47.869648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:47.869648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:51:47.869648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:51:47.870824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:47.904995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 747/1497] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:51:47 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:51:47.909188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:47.919637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987656a2-af32-4c85-8379-6e5fac0f944f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:47.974054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:47.974869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:47.975154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:47.975431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:47.976146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:47.977168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:47.977168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:51:47.977168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:51:47.978360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:47.998594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 751/1498] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:51:47 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:51:48.002641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.013388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec86b0-8f80-4232-a585-f185cbbf3350 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.017757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.018410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.018693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.018970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.019695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.020943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:48.020943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:51:48.020943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:51:48.021910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.034344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 748/1499] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:51:48.038712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.048505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f473cf-f66d-4480-acdf-b01f7126f718 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.053605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.054258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.054542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.054819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.055552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.056609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:48.056609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:51:48.056609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:51:48.057506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434d281-25be-4725-a6a2-414d01667bbe admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.070494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 752/1500] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:51:48.074511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.084547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15776f-282e-4661-bb1e-269182eb3bd4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.089009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.089886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.090174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.090456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] PATH_INFO: `/v3/projects/b56098f3defb49c28ced46944a5edb12/users/21f7516a51e24c5da4853ca38fdb5a39/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.105258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56098f3defb49c28ced46944a5edb12, user_id=21f7516a51e24c5da4853ca38fdb5a39, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.106298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:48.106298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:51:48.106298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:51:48.108257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e52294-5cc4-4455-9f94-60e9ca12e56c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.121994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 749/1501] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:51:48 2022] PUT /identity/v3/projects/b56098f3defb49c28ced46944a5edb12/users/21f7516a51e24c5da4853ca38fdb5a39/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:51:48.133510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ddc0-d68b-4abe-ae2b-2243da51d85b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.133804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ddc0-d68b-4abe-ae2b-2243da51d85b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.134082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1ddc0-d68b-4abe-ae2b-2243da51d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.160835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-79f1ddc0-d68b-4abe-ae2b-2243da51d85b None None] MFA Rules not processed for user `21f7516a51e24c5da4853ca38fdb5a39`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:51:48.162832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-79f1ddc0-d68b-4abe-ae2b-2243da51d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:48.227226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 753/1502] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:51:48.241140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc67e2-2a27-4f7c-bd37-75bdf628151b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.241440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc67e2-2a27-4f7c-bd37-75bdf628151b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.241725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc67e2-2a27-4f7c-bd37-75bdf628151b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.243129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 750/1503] 10.222.0.24 () {54 vars in 963 bytes} [Tue May 24 13:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:51:48.247985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cfde-8206-42f6-921e-9ead8a2f9f50 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.248275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cfde-8206-42f6-921e-9ead8a2f9f50 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.248554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cfde-8206-42f6-921e-9ead8a2f9f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.267551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9044cfde-8206-42f6-921e-9ead8a2f9f50 None None] MFA Rules not processed for user `9da57903126d421b8ac6c0e26b3fb3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:51:48.268701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9044cfde-8206-42f6-921e-9ead8a2f9f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:48.314445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:51:48.323692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 754/1504] 10.222.0.24 () {58 vars in 1060 bytes} [Tue May 24 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:51:48.331283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc67e2-2a27-4f7c-bd37-75bdf628151b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.342158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc67e2-2a27-4f7c-bd37-75bdf628151b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.345654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.346290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.346577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.346856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.355518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.357088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.366539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 751/1505] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:51:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:51:48.407691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f19b4e-3d3e-4d17-974f-b21fe7b375e8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.408020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f19b4e-3d3e-4d17-974f-b21fe7b375e8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.408308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f19b4e-3d3e-4d17-974f-b21fe7b375e8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.409589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 755/1506] 10.222.0.24 () {52 vars in 829 bytes} [Tue May 24 13:51:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:51:48.413234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.475018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e7fb5-5935-4579-9ce7-6a02d9164a66 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.479479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.480374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.480666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.480948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] PATH_INFO: `/v3/users/21f7516a51e24c5da4853ca38fdb5a39` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.482124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] RBAC: Authorizing `identity:delete_user(user_id=21f7516a51e24c5da4853ca38fdb5a39)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.483145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:48.483145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:51:48.483145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:51:48.484402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.550081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21f7516a51e24c5da4853ca38fdb5a39'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:48.550648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21f7516a51e24c5da4853ca38fdb5a39'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:48.570597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21f7516a51e24c5da4853ca38fdb5a39'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:48.571396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21f7516a51e24c5da4853ca38fdb5a39'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:48.583715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21f7516a51e24c5da4853ca38fdb5a39'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:48.587652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:51:48.600740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 752/1507] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:51:48 2022] DELETE /identity/v3/users/21f7516a51e24c5da4853ca38fdb5a39 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:51:48.605181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.605479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.605769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.623514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:51:48.624751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:48.683112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 756/1508] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:51:48.692963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:48.694368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:48.752396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35cebb-3e29-4220-a1b2-dd9942a4189f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:48.756089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:48.756990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:48.757276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:48.757551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:48.766207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:48.767642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7befe-f4a3-484e-a471-c533cf2a3dd7 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:48.776787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 753/1509] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:51:49.125470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:51:49.127019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:51:49.139491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d563be-1957-4617-9b15-45941f21d224 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:51:49.144243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:51:49.145109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:51:49.145957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:51:49.146347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] PATH_INFO: `/v3/projects/b56098f3defb49c28ced46944a5edb12` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:51:49.147732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56098f3defb49c28ced46944a5edb12)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:51:49.148796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:51:49.148796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:51:49.148796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:51:49.150012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:51:49.210465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56098f3defb49c28ced46944a5edb12', 'request_id': 'req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:49.223654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56098f3defb49c28ced46944a5edb12', 'request_id': 'req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:49.230348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56098f3defb49c28ced46944a5edb12', 'request_id': 'req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:49.244531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] The token cache is being invalidate because project b56098f3defb49c28ced46944a5edb12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:51:49.244825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:49.245547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b72dc7c5-09fe-4aed-ba49-4e5c74475e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:51:49.247118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 757/1510] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:51:49 2022] DELETE /identity/v3/projects/b56098f3defb49c28ced46944a5edb12 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:51:58.772792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:51:59.241464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:53:29.587227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:29.587783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:29.587783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:29.589215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:29.594058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b22b0-fe31-48e7-879d-da0c38b0dcdb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:29.594521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b22b0-fe31-48e7-879d-da0c38b0dcdb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:29.594630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b22b0-fe31-48e7-879d-da0c38b0dcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:29.613803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-747b22b0-fe31-48e7-879d-da0c38b0dcdb None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:29.615022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-747b22b0-fe31-48e7-879d-da0c38b0dcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:29.672359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:29.842996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:29.844685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:29.925230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9e16b-4520-4b24-bf36-adb999553152 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:29.930438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:29.931084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:29.931372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:29.931651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:29.939999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:29.941582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc9cc65-426d-4da7-9db5-f0ac02564998 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:29.950690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 755/1513] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:30.852698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf43c-ba65-4523-9f68-f59b2198f11a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:30.853245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf43c-ba65-4523-9f68-f59b2198f11a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:30.853245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cf43c-ba65-4523-9f68-f59b2198f11a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:30.854667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:30.859594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce5fd5-5064-458c-883d-5a717d404a77 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:30.860046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce5fd5-5064-458c-883d-5a717d404a77 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:30.860191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce5fd5-5064-458c-883d-5a717d404a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:30.879107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90ce5fd5-5064-458c-883d-5a717d404a77 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:30.880364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90ce5fd5-5064-458c-883d-5a717d404a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:30.888560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:31.044749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cf43c-ba65-4523-9f68-f59b2198f11a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:31.059150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cf43c-ba65-4523-9f68-f59b2198f11a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:31.065020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:31.065653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:31.065945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:31.066228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:31.074220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:31.075857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d120302-b754-41c0-a832-ac9a886c587e service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:31.085115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 760/1516] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:31.278385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aa7e0-be79-47ce-b3f0-667256156f69 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:31.278672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aa7e0-be79-47ce-b3f0-667256156f69 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:31.278953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624aa7e0-be79-47ce-b3f0-667256156f69 None None] PATH_INFO: `/v3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:31.280319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 757/1517] 10.222.0.55 () {54 vars in 925 bytes} [Tue May 24 13:53:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:53:31.286662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee7c3-3877-4723-a045-2763268c81eb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:31.286963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee7c3-3877-4723-a045-2763268c81eb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:31.287250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ee7c3-3877-4723-a045-2763268c81eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:31.305901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-403ee7c3-3877-4723-a045-2763268c81eb None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:31.307158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-403ee7c3-3877-4723-a045-2763268c81eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:31.367435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 761/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Tue May 24 13:53:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:35.256808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dbb5-10a5-4b89-a318-c6bdb7a35463 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:35.257237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dbb5-10a5-4b89-a318-c6bdb7a35463 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:35.257378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c1dbb5-10a5-4b89-a318-c6bdb7a35463 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:35.258823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 758/1519] 10.222.0.24 () {54 vars in 960 bytes} [Tue May 24 13:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:35.263578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa49d-4849-466d-98f4-7b77a4707526 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:35.264004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa49d-4849-466d-98f4-7b77a4707526 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:35.264192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fa49d-4849-466d-98f4-7b77a4707526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:35.284154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4e3fa49d-4849-466d-98f4-7b77a4707526 None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:35.285376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e3fa49d-4849-466d-98f4-7b77a4707526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:35.293735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 762/1520] 10.222.0.24 () {58 vars in 1057 bytes} [Tue May 24 13:53:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:35.301300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1dbb5-10a5-4b89-a318-c6bdb7a35463 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:35.314750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c1dbb5-10a5-4b89-a318-c6bdb7a35463 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:35.319558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:35.320248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:35.320533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:35.320810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:35.328786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:35.331053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a99d5-5326-4be8-b36f-c0f63aabaabb service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:35.340340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 759/1521] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:53:42.330559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923e00-d818-4730-842c-57f401a028fd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:42.331186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923e00-d818-4730-842c-57f401a028fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:42.331186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd923e00-d818-4730-842c-57f401a028fd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:42.332542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:42.337495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b835-51d3-4cbc-8a7c-b8a8523d68a9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:42.337947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b835-51d3-4cbc-8a7c-b8a8523d68a9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:42.338082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b835-51d3-4cbc-8a7c-b8a8523d68a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:42.356637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-edf3b835-51d3-4cbc-8a7c-b8a8523d68a9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:42.357846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-edf3b835-51d3-4cbc-8a7c-b8a8523d68a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:42.365956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:42.521119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd923e00-d818-4730-842c-57f401a028fd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:42.535478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd923e00-d818-4730-842c-57f401a028fd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:42.540754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:42.541387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:42.541671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:42.541947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:42.550115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:42.551708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adc6a10-eb31-492e-b1d1-0f221d6fcf5b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:42.560804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 764/1524] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:43.439900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765556-2e0d-4bd2-a25c-2d04527852de None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:43.440458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765556-2e0d-4bd2-a25c-2d04527852de None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:43.440458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3765556-2e0d-4bd2-a25c-2d04527852de None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:43.441889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 13:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:43.446648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1b6cd-c02b-4847-9a59-b06de7b34133 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:43.447089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1b6cd-c02b-4847-9a59-b06de7b34133 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:43.447257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb1b6cd-c02b-4847-9a59-b06de7b34133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:43.465940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0bb1b6cd-c02b-4847-9a59-b06de7b34133 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:43.467155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0bb1b6cd-c02b-4847-9a59-b06de7b34133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:43.475204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 13:53:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:43.635226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765556-2e0d-4bd2-a25c-2d04527852de None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:43.649429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3765556-2e0d-4bd2-a25c-2d04527852de None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:43.654801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:43.655444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:43.655734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:43.656057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:43.664425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:43.666050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdb7b2-7d5c-4c3a-9744-188692425215 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:43.675799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 762/1527] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:48.278853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:48.279435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:48.279435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:48.280824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:48.287016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2130647-fd75-4403-b336-978d67b13545 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:48.287434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2130647-fd75-4403-b336-978d67b13545 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:48.287580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2130647-fd75-4403-b336-978d67b13545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:48.307489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c2130647-fd75-4403-b336-978d67b13545 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:48.308713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c2130647-fd75-4403-b336-978d67b13545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:48.316632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:48.352836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:48.354509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:48.411034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868b3b93-f2b5-4a5e-b2a8-9f11ff31ef8b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:48.414599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:48.415252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:48.415544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:48.415837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:48.424627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:48.426075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2e30b-3972-4d8f-9a80-6804bfed9a97 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:48.430890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:53:48.436095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 767/1530] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:50.645306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedb8a4-c757-4ce3-bbb9-11e277dee3c1 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:50.645836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedb8a4-c757-4ce3-bbb9-11e277dee3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:50.645948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedb8a4-c757-4ce3-bbb9-11e277dee3c1 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:50.647258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:50.652423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114171f-089c-42a3-8322-95a64b10631f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:50.652830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114171f-089c-42a3-8322-95a64b10631f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:50.652997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114171f-089c-42a3-8322-95a64b10631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:50.671963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d114171f-089c-42a3-8322-95a64b10631f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:50.673188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d114171f-089c-42a3-8322-95a64b10631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:50.680875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:50.715846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedb8a4-c757-4ce3-bbb9-11e277dee3c1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:50.727029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedb8a4-c757-4ce3-bbb9-11e277dee3c1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:50.730568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:50.731212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:50.731504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:50.731789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:50.740880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:50.742313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d47c2-fd26-43da-b5ea-ed22b5d227e1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:50.752313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 765/1533] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:54.424380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:54.424974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:54.424974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:54.426380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 13:53:54.431556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51a9f5-a0a1-4948-a011-f149d6c69668 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:54.431999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51a9f5-a0a1-4948-a011-f149d6c69668 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:54.432157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b51a9f5-a0a1-4948-a011-f149d6c69668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:54.450301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5b51a9f5-a0a1-4948-a011-f149d6c69668 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:54.451497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b51a9f5-a0a1-4948-a011-f149d6c69668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:54.459158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:54.730094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:54.731750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:54.788716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a87f9b-f7e1-4d22-86a0-7c02df5fa81e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:54.792237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:54.792885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:54.793172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:54.793453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:54.802397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:54.803843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653170e3-d186-4b5d-882d-e75f9d62cd9d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:54.813736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 770/1536] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:53:57.033358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:57.033914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:57.033914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:57.035345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 13:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:53:57.040478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3f66-3e25-4728-afd1-98fb95a82753 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:57.040898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3f66-3e25-4728-afd1-98fb95a82753 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:57.041055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a3f66-3e25-4728-afd1-98fb95a82753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:57.058952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fd0a3f66-3e25-4728-afd1-98fb95a82753 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:53:57.060253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fd0a3f66-3e25-4728-afd1-98fb95a82753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:57.068517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 13:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:53:57.343602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:53:57.345550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:53:57.359021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91a54e-8d34-4e02-8e6f-44cf4de61ae4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:53:57.363134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:53:57.363863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:53:57.364216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:53:57.364535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:53:57.373601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:53:57.375028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34434f95-3afd-4607-aacc-5f3f9eb95d6b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:53:57.385302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 768/1539] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:54:07.053754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=32 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:54:07.381206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=70 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:29.006529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70985e9-21ae-4835-8970-c2cbb93ec015 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.007230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70985e9-21ae-4835-8970-c2cbb93ec015 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.007230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70985e9-21ae-4835-8970-c2cbb93ec015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.035834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a70985e9-21ae-4835-8970-c2cbb93ec015 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.037058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a70985e9-21ae-4835-8970-c2cbb93ec015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.049868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 772/1540] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:29.057790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088bfd-4c64-45f9-b16a-bb9dc107f58b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.058897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088bfd-4c64-45f9-b16a-bb9dc107f58b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.059107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31088bfd-4c64-45f9-b16a-bb9dc107f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.061295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56351ff6-9a9d-4574-95bd-b03b3f8f436b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.061588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56351ff6-9a9d-4574-95bd-b03b3f8f436b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.061874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56351ff6-9a9d-4574-95bd-b03b3f8f436b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.088554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-56351ff6-9a9d-4574-95bd-b03b3f8f436b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.088931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-31088bfd-4c64-45f9-b16a-bb9dc107f58b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.089928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-56351ff6-9a9d-4574-95bd-b03b3f8f436b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.090453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-31088bfd-4c64-45f9-b16a-bb9dc107f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.102893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 769/1541] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.104238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfdea0-fcda-4ef7-a8bb-246b6d17d4ed None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.104543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfdea0-fcda-4ef7-a8bb-246b6d17d4ed None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.105336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cfdea0-fcda-4ef7-a8bb-246b6d17d4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.107714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 773/1542] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:29.113893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322780-74d0-42b7-bc2a-59ebded150df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.114183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322780-74d0-42b7-bc2a-59ebded150df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.114463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91322780-74d0-42b7-bc2a-59ebded150df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.145073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-53cfdea0-fcda-4ef7-a8bb-246b6d17d4ed None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.146471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-91322780-74d0-42b7-bc2a-59ebded150df None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.146609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-53cfdea0-fcda-4ef7-a8bb-246b6d17d4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.147651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91322780-74d0-42b7-bc2a-59ebded150df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.162771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 770/1543] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.164145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 774/1544] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:29.164282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.164481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.164733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.165505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.165796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.166083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.197937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.199375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.200895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.202102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.214684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 775/1545] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.215690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.220537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 771/1546] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.221400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.233956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad131b3-9a75-43a4-b925-49e4d3cc9c73 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.235998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdda65-0dd1-463c-b6cf-1fad855b9858 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.243371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.244074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.244365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.244650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.247067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.247451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.247572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.248193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.248481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.248761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.249162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b913aa24-3fbd-46ab-becc-6231debdf954 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.249510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.249823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.250850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:29.250850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:55:29.250850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:55:29.251947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.263499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 776/1547] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.265361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140d27-34fd-4931-ae3c-64c55f27022d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.265836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140d27-34fd-4931-ae3c-64c55f27022d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.266286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee140d27-34fd-4931-ae3c-64c55f27022d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.266815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 772/1548] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.273763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.281439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcbfe4-c845-401a-a84d-5e63b7974aca admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.286540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.287283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.287666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.288019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.288760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.289073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.290857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cadc6-2f9a-4909-96df-9c47eeeba1ba admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.293563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ee140d27-34fd-4931-ae3c-64c55f27022d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.294788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee140d27-34fd-4931-ae3c-64c55f27022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.303267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 777/1549] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.304689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.304984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.305264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.306770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 773/1550] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.308360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1c15-c01a-4149-a998-df39d0616284 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.308913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1c15-c01a-4149-a998-df39d0616284 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.309485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505f1c15-c01a-4149-a998-df39d0616284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.329395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-505f1c15-c01a-4149-a998-df39d0616284 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.330639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-505f1c15-c01a-4149-a998-df39d0616284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.330762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.332089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.338736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 774/1551] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.340401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 778/1552] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.340770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.341067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.341346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.344713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.355375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb492f-4bdf-4773-a0d1-88f03de61ab6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.361822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.362624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.362974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.363457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.363936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.364244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.364715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.365787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:29.365787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:55:29.365787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:55:29.367061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5a9c0-c8ac-4522-b8ea-09fea581b61d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.372945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 775/1553] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.373741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.385852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51932e1e-ba07-4601-8a5c-f2c61b4e53f5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.391671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.392379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.392671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.392954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.393656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.395567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.396408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 779/1554] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.397706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bbb5-0d17-41d0-8a15-484880805bec None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.398011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bbb5-0d17-41d0-8a15-484880805bec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.398296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837bbb5-0d17-41d0-8a15-484880805bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.416907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9837bbb5-0d17-41d0-8a15-484880805bec None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.418688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9837bbb5-0d17-41d0-8a15-484880805bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.427871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 776/1555] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.427871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.487053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 780/1556] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.488418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.488715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.489000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.494543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e3bed2-2004-4121-8798-cb9b9defb6f1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.499516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.500216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.500503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.500785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.501475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.503412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8fc8b-341c-466f-9e7e-a96f31942f45 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.508811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.510022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.518416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 781/1557] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.519141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.529653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f862b9-cc22-40d0-ac54-1f8bbe92e5fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.534365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.535026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.535318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.535601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.539180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 777/1558] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.539334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.539334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.539495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.539495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.539495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.539737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee481e4-4461-40e7-8c15-6a1f4633ca43 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.558883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 782/1559] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.558883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.558883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.558883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.559376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.559376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.571065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.572341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.619574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 778/1560] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.620887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.626569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:29.630862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985fea1-4713-41c7-9216-c4e6fd04b6fd None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.635755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 783/1561] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.637874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.638649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.639571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.639907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.640231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.640934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.641994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:29.641994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:55:29.641994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:55:29.643198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.647510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf574b6-a97b-435b-ba9e-a997df163b94 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.652790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.653454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.653748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.654025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.654717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.657076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b3d06-9f8a-471c-b5f1-5b9ececabd03 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.663937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:29.668217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 779/1562] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/users => generated 297 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.669309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.679422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587a4b-da67-4a00-b9a4-72d6dbd5a282 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.682948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 784/1563] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.682948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.682948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.682948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.686357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.687009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.687298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.687578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.688349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.688660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.690264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.699698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:29.700938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:29.702563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 780/1564] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.703493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.713490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e2fc5-83e8-4106-bd3b-e9da71efbc22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.715269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 785/1565] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:29.716124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.718952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.719605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.719918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.720198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.721105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.723572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.725831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888bccd5-83fb-46da-a45c-a5f61e255af7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.730648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.731322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.731610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.731927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.732622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.734632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.744517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 781/1566] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.749015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.759395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa96dc-790b-428c-a5ec-b6c001ae0317 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.762150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 786/1567] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.764339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.764996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.765448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.765578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.766179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.766447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.766760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.768410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.781542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 782/1568] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.784703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.794258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cb5924-b50e-462b-9633-63c1f1438fef admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.827211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1817c3-b526-4c6f-9647-0c5d7b12641b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.831842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.832516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.832806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.833095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.833826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.834139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.835746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.847151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 787/1569] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.848004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.852814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.854207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.854584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.855445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.856571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.858213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503313a-4307-47fa-9c0b-fb17ad7eae69 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.858365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.862862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.863520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.863837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.864127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.864867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.866438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.872649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 783/1570] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:29.872649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.878274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 788/1571] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:29.879037 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.885241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e86b61d-6e69-4f94-a0a3-2202f8eed7f4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.889087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bced9c-c0fd-4686-867a-760589b13ebd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.890865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.891538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.891812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.892128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.892847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.893785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.894497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.894751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.894930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.895284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.896113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:29.896498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.898214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.910339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 789/1572] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:29.911171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.921872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2a886-4aad-4d15-a7b4-d12871a6e94f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.926852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:29.927577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:29.927973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:29.928323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:29.932191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 784/1573] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:29.932348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:29.932447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.932558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:29.941028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 790/1574] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:29.941884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:29.952281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43026500-d101-4035-be17-c9f52283dc78 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.994319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2128d17-15ae-4121-ba04-8764954fedd1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:29.999581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.000363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.000724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.001070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.001870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.003877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.009768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.010422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.010707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.011015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.011958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.013843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.024393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 785/1575] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.028554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.038602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e17e16-977f-4d92-b414-1849eb51f881 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.040271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 791/1576] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:29 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.041004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.046030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.049059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.101494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.102189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.102484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.102767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.103470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.105458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.108647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeabbeb-6887-401f-97d1-31a5301dff06 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.113277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.113947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.114239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.114531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.115265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.116912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.128291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 792/1577] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.129192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.139101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 786/1578] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.139101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.141848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053d478-763a-4aed-bad6-3be492549b9e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.197679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e2158-1add-47f8-bdc8-30456851c1a5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.198108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.199523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.199953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.200242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.200981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.202602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.203040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.203706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.204030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.204313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.205003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.206877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.212785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 793/1579] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.213630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.225403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb30becd-aad7-4a3c-9769-1b4f94735f2f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.230680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.231344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.231637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.231955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.232696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.236285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 787/1580] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.236285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.236521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.258842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 794/1581] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.258842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.267045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cffb9-0fc0-4084-8f56-268d2f02dc95 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.295215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f791e-c82d-44b2-bfc8-3757e28a7eb8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.299786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.300466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.300754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.301049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.301795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.303617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.317536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 788/1582] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.317536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.325439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbed0c6-4f0d-4939-bd52-5f8485e15e41 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.327040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.327721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.328050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.328336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.329081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.330074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.330707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.330821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.330998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.331282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.332032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.333914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.343317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 795/1583] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.343317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.344151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.354555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213fd5-2452-448f-8b89-97f5c6ca587c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.357126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 789/1584] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/users => generated 302 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.359855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.361107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.361107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.361340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.361451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] PATH_INFO: `/v3/projects/54ee7d13408645e6b6517f541fe45670/users/532b36f272dc432aad6faa294bd1bc6c/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.376013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ed34d-1365-4e97-b2f4-346508f1c033 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.376275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.381252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.381932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.382077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ee7d13408645e6b6517f541fe45670, user_id=532b36f272dc432aad6faa294bd1bc6c, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.382290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.382570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.383265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.383362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:30.383362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:55:30.383362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:55:30.385192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.385662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.400690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 796/1585] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:30 2022] PUT /identity/v3/projects/54ee7d13408645e6b6517f541fe45670/users/532b36f272dc432aad6faa294bd1bc6c/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:30.401578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.410020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 790/1586] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/users => generated 282 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.410832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.420630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d668c1f-ca74-468a-893c-727f029154f5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.468059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64ae8-fcf1-4cbd-935f-ef240c9cbbc7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.472846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.473729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.474257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.474257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] PATH_INFO: `/v3/projects/469bcf4beb3041559397e70538a7ea12/users/fae3b6e3b2e24719ad90614dc5c7e431/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.480639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.481270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.481576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.481843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.482537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.484937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.489398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=469bcf4beb3041559397e70538a7ea12, user_id=fae3b6e3b2e24719ad90614dc5c7e431, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.492085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.508793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 797/1587] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:30 2022] PUT /identity/v3/projects/469bcf4beb3041559397e70538a7ea12/users/fae3b6e3b2e24719ad90614dc5c7e431/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:30.509617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.512411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 791/1588] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:30.513282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.522735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b35c9c-a147-4b59-9836-5985b8573c21 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.576446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7244468-e4c9-4504-b473-0b3184a03cb9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.579891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.580954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.581092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.581458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.581624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.581731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] PATH_INFO: `/v3/projects/47cda95585534d33aebb09e4aadfb1fb/users/2f00eaae5afb48808260a8836f5b4d27/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.581916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.585732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.585732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.585732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.597734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 798/1589] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.598667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.601310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47cda95585534d33aebb09e4aadfb1fb, user_id=2f00eaae5afb48808260a8836f5b4d27, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.608243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.609197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66f966-7ec1-4910-a6c3-94e16c54683f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.614183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.614968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.615348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.615722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.616526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.618202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0c7c03-36bf-4d35-a6f9-035b34af654a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.624651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 792/1590] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:30 2022] PUT /identity/v3/projects/47cda95585534d33aebb09e4aadfb1fb/users/2f00eaae5afb48808260a8836f5b4d27/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:30.625571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.632761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 799/1591] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.632761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9257b-8c7f-4ce5-9d32-cba0276765e8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.632761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9257b-8c7f-4ce5-9d32-cba0276765e8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.632761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9257b-8c7f-4ce5-9d32-cba0276765e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.652013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-06b9257b-8c7f-4ce5-9d32-cba0276765e8 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:30.653882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-06b9257b-8c7f-4ce5-9d32-cba0276765e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:30.684415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.696543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3415aae-4858-4328-a0dc-f98d03304e19 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.701708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.702373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.702660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.702939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.703668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.705286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.707316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:30.716513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 793/1592] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.718236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 800/1593] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:30.719548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.719866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.720155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.722157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.732657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9823b26-a095-42db-8c68-848be4037495 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.737142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.737803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.738091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.738254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:30.738402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.739091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.739462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:30.740701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.748175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 801/1594] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:30.748981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.751890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 794/1595] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.752722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.758587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aee150-96c7-4cfa-99b7-9e5c99316c21 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.763580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ddf3e-056a-4df7-b0c2-b3b940fc7d51 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.764383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.765061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.765409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.765694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.766482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.768144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.768289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.768959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.769246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.769604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.770331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.771912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c619b08-82b0-4299-aecd-83a41238ba62 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.782112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 802/1596] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.782112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.785522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 795/1597] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.785522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.785522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.785522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.790435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2de605-5a41-4c62-8f88-c06c085e81c8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.795576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.796482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.796770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.797048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] PATH_INFO: `/v3/projects/3db44af3db3441b7a0c5eb524eb35806/users/3eb0459bdac1456a900046e264835a6c/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.811221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] MFA Rules not processed for user `2f00eaae5afb48808260a8836f5b4d27`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:30.811941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db44af3db3441b7a0c5eb524eb35806, user_id=3eb0459bdac1456a900046e264835a6c, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.812474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:30.814611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.828292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 803/1598] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:30 2022] PUT /identity/v3/projects/3db44af3db3441b7a0c5eb524eb35806/users/3eb0459bdac1456a900046e264835a6c/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:30.829190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.876967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 796/1599] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:30.877955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.896276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f50131-c139-4856-8fa4-ba11118e594d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.902268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.903381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.904364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.904888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.906264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.910379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.924982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 804/1600] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.925920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.937928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6352c5b-33e5-44a7-95b5-8901df229b38 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.943185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708887e4-c9c0-4ee4-afe4-ddf850718280 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:30.944185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.945315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:30.945817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:30.946296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:30.947641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:30.950763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:30.961337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 805/1601] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:30.962168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:30.999136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:30.999789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.000118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.000403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.009677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.011233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.018314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d66c-ffcd-459d-aba9-a80c84d0b942 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.020807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 797/1602] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:31.021611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.022070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.022758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.022993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.023274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.031261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c2d7f-bb92-49eb-be9a-17579af50ac0 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.034256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.035703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.036226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.036728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.037020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.037305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.038040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.039655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209374f-b4c7-482a-89d3-76bb031c9dfb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.046617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 806/1603] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:31.047259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.055754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 798/1604] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:31.057000 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a79b843-4815-42f9-a696-98fb3d437aae service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.057161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afcf2e-0068-4552-8ecd-bd407ba7807a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.057437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afcf2e-0068-4552-8ecd-bd407ba7807a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.057718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55afcf2e-0068-4552-8ecd-bd407ba7807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.063281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.064009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.064499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.064751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:31.064884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] PATH_INFO: `/v3/projects/2ad3abc8cf774b279c6c68c27a1877d3/users/0e2bd48d78a540d6978f128f9af94e0a/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.077719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55afcf2e-0068-4552-8ecd-bd407ba7807a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.079026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55afcf2e-0068-4552-8ecd-bd407ba7807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.082878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad3abc8cf774b279c6c68c27a1877d3, user_id=0e2bd48d78a540d6978f128f9af94e0a, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.085740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.087957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 799/1605] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.089947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d0fb4-12ae-48b1-9a08-cad69b58213c None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.090148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d0fb4-12ae-48b1-9a08-cad69b58213c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.090433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d0fb4-12ae-48b1-9a08-cad69b58213c None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.091798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 800/1606] 10.222.0.24 () {54 vars in 961 bytes} [Tue May 24 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:31.092831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d0fb4-12ae-48b1-9a08-cad69b58213c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.109104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 807/1607] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:31 2022] PUT /identity/v3/projects/2ad3abc8cf774b279c6c68c27a1877d3/users/0e2bd48d78a540d6978f128f9af94e0a/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:31.109364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d0fb4-12ae-48b1-9a08-cad69b58213c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.110058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.145968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:31.198118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d391c9-126a-4c0b-95a8-1e0bd2944baf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.201844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.202874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.203790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.203790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] PATH_INFO: `/v3/projects/7eda19672b7c450d8c3b6df5aa3abe21/users/2e5e815227ff47f485db253b984509f4/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.206721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.207871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.208318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.208451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] PATH_INFO: `/v3/projects/4fd016cc76ae4093afd1a12d340ddc65/users/7c0f2d7211af44a3a39e4da9613bebec/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.222932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eda19672b7c450d8c3b6df5aa3abe21, user_id=2e5e815227ff47f485db253b984509f4, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.225652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.227223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd016cc76ae4093afd1a12d340ddc65, user_id=7c0f2d7211af44a3a39e4da9613bebec, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.230312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.242977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 801/1608] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:31 2022] PUT /identity/v3/projects/7eda19672b7c450d8c3b6df5aa3abe21/users/2e5e815227ff47f485db253b984509f4/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:31.244167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.244968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 808/1609] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:31 2022] PUT /identity/v3/projects/4fd016cc76ae4093afd1a12d340ddc65/users/7c0f2d7211af44a3a39e4da9613bebec/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:31.247217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.248753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.326839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819aba-640e-4a81-834a-7a8a9877e025 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.331282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221da8e9-7b43-46ef-9849-fd8cb1e9a31f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.331443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.332090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.332456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.332746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.409036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.410716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549e492-d7b0-4a9d-9082-311e2940000f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.411695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.412699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.413058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.413565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] PATH_INFO: `/v3/projects/002783353d9248ea837d793d9f5df662/users/04bb2299091f45e8a53477aee4255e88/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.423502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 809/1610] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:31.425090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd482c-ad16-4acc-a70b-f64875e6f0fb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.425449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd482c-ad16-4acc-a70b-f64875e6f0fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.425799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd482c-ad16-4acc-a70b-f64875e6f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.445369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b2cd482c-ad16-4acc-a70b-f64875e6f0fb None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.446810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b2cd482c-ad16-4acc-a70b-f64875e6f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.451835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=002783353d9248ea837d793d9f5df662, user_id=04bb2299091f45e8a53477aee4255e88, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.455885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edf567-dafc-4d97-b836-45e8a6d8dcf2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.486176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 802/1611] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:31 2022] PUT /identity/v3/projects/002783353d9248ea837d793d9f5df662/users/04bb2299091f45e8a53477aee4255e88/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:31.487860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6adc7-9baa-41aa-89b8-a3df2cfb993a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.490738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6adc7-9baa-41aa-89b8-a3df2cfb993a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.490738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6adc7-9baa-41aa-89b8-a3df2cfb993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.526517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:31.546988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-abe6adc7-9baa-41aa-89b8-a3df2cfb993a None None] MFA Rules not processed for user `0e2bd48d78a540d6978f128f9af94e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.551458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:31.552456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abe6adc7-9baa-41aa-89b8-a3df2cfb993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.580224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 810/1612] 10.222.0.24 () {58 vars in 1058 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.581987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51d14d-2bb1-4346-b24c-b99db45e85b8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.582378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51d14d-2bb1-4346-b24c-b99db45e85b8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.583297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51d14d-2bb1-4346-b24c-b99db45e85b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.619800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ce51d14d-2bb1-4346-b24c-b99db45e85b8 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.621608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce51d14d-2bb1-4346-b24c-b99db45e85b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.669564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 803/1613] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.671455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.671852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.672216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.701136 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 811/1614] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.702848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.703331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.703863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.713381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] MFA Rules not processed for user `7c0f2d7211af44a3a39e4da9613bebec`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.716653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.766829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] MFA Rules not processed for user `04bb2299091f45e8a53477aee4255e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:31.768125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:31.825507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 804/1615] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.832601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.891784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 812/1616] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:31.892761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:31.905593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822d5e1-03f7-419e-bf23-3a7a5f34b9d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:31.911516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:31.912244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:31.912545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:31.912828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:31.913541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:31.915977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:31.981080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 813/1617] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:31 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:31.981080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.003164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50571f7-4dea-4c02-afa5-75119857ecc3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.043321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:32.059423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624bac95-59a3-43d8-b7ab-fb63ccd1a285 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.064568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.066052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.066530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.066963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.082294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:32.135726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.139212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.139835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.140742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.201149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.203080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.217918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 814/1618] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:32.219257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.233601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d54cf6-cfc8-4ea3-ba1f-e7b287600bd9 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.241896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.243963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.244496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.244929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.254429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.258581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.296467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.298655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.313630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 805/1619] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:32.314912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.337686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a805fd1-e8e6-4791-aa25-0081ad37495b service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.337867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 815/1620] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:32.341233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.482614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aaf71-a50d-47be-836a-571246d2ba31 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.491152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.491910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.492212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.492502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.493225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.496650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.535258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.535981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.538999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.539134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 816/1621] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:32.539134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.539440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.540234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.542528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c3229-058f-4fe4-8575-88a134aa3a4d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.562923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e1b84-b0fa-4605-bb66-09f09f98236a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.583358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.584092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.584494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.584788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.585549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.587607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.642991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 817/1622] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:32.644079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.652505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 806/1623] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/projects => generated 403 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:32.655392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52372-ce27-4563-80fd-cf7c160c5318 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.657413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52372-ce27-4563-80fd-cf7c160c5318 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.657740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52372-ce27-4563-80fd-cf7c160c5318 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.661228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 807/1624] 10.222.0.24 () {54 vars in 912 bytes} [Tue May 24 13:55:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:32.663150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52372-ce27-4563-80fd-cf7c160c5318 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.670604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e909757-6b0d-461f-a1e0-f8450de52836 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.798549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:32.803784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:32.830635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.831641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.831972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.832420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.833458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.835996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c088634-f0ce-4294-b86b-cdfe72e5e7fe admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.857847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52372-ce27-4563-80fd-cf7c160c5318 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.859222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 818/1625] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:32.863410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.863985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.864509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.868888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.869750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.870410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.870410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.871325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.874679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:32.901526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 808/1626] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:32.903471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:32.930265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] MFA Rules not processed for user `fae3b6e3b2e24719ad90614dc5c7e431`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:32.931533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:32.948506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df489a1f-6a14-46e8-b373-797eca002524 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:32.956580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:32.957566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:32.957736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:32.958946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:32.959858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:32.962115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02a6b8-7f72-4128-a6ee-5691121ffbfd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.007631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 809/1627] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/users => generated 308 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:33.007631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.007631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.007631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.052254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:33.053711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:33.080650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 819/1628] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:33.094318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.129373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63106a62-2155-4132-97fd-5479545c91c6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.139838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.140899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.141191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.141478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] PATH_INFO: `/v3/projects/77599bb8215e49e1b498591af0eaf977/users/107a1c486be94ca3a0b542341ab35921/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.194569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77599bb8215e49e1b498591af0eaf977, user_id=107a1c486be94ca3a0b542341ab35921, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.199669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.222634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 820/1629] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:33 2022] PUT /identity/v3/projects/77599bb8215e49e1b498591af0eaf977/users/107a1c486be94ca3a0b542341ab35921/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:33.224307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.254008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 810/1630] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:55:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:33.256026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.286428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059e0bd-84f6-4e1e-b03e-b6ce285b4b05 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.350650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ce17-e3f3-4c4f-abef-3590d2236a08 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.357191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.358124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.358997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.358997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] PATH_INFO: `/v3/projects/ee3362848f30450f8cb418b07e70e35f/users/bbefbc57f5a34859a75bd147b211c06f/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.381463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3362848f30450f8cb418b07e70e35f, user_id=bbefbc57f5a34859a75bd147b211c06f, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.387325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397b28b-1293-4f8d-a39e-09a504db71f2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.411119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.411902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 821/1631] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:33 2022] PUT /identity/v3/projects/ee3362848f30450f8cb418b07e70e35f/users/bbefbc57f5a34859a75bd147b211c06f/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:33.412041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.412461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.412827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.413160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.413456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.413733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.413990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.416929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.440102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 811/1632] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:33.441271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.446449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] MFA Rules not processed for user `fae3b6e3b2e24719ad90614dc5c7e431`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:33.447965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:33.454817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3c56a-822c-4dea-bd33-8ede6bddc905 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.499512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:33.540488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:33.546626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 822/1633] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:33.547579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.554780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.556068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.556411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.556638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.559484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.566466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.586761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 812/1634] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:33.588323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.612217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36c90e-2a62-407c-a5ac-889cbd500e97 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.624944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.626028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.626412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.626676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.628172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.630896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da0315-9089-485a-9498-99ce3b9183aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.655094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 813/1635] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:33.657217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.657610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.657974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.659191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bcbe9-eeaa-414d-be62-9f5068243622 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.664918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.665602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.665895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.666235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.692034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] MFA Rules not processed for user `2e5e815227ff47f485db253b984509f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:33.693385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:33.782248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 814/1636] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:33.782248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.784221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.787008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753cfa1-8d77-4ddc-93e4-aaa3c8c10adb service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.802355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6f02f-b574-43b5-b441-98cc3ca5be42 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.803320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 823/1637] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:33.804862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f17c-0006-4d29-924a-0dc920ced056 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.805264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f17c-0006-4d29-924a-0dc920ced056 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.805750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499f17c-0006-4d29-924a-0dc920ced056 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.807545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 824/1638] 10.222.0.24 () {54 vars in 961 bytes} [Tue May 24 13:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:33.808433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f17c-0006-4d29-924a-0dc920ced056 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.825579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499f17c-0006-4d29-924a-0dc920ced056 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.829251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.830315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.830612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.830902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] PATH_INFO: `/v3/projects/f6625f5c82d84fc49bc92ded91365c7c/users/33941ba8581e4385ae7032d61d77be58/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.835942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:33.837118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:33.837730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:33.839298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] PATH_INFO: `/v3/projects/77599bb8215e49e1b498591af0eaf977/users/107a1c486be94ca3a0b542341ab35921/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:33.844070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77599bb8215e49e1b498591af0eaf977, user_id=107a1c486be94ca3a0b542341ab35921, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.848237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.860764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6625f5c82d84fc49bc92ded91365c7c, user_id=33941ba8581e4385ae7032d61d77be58, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:33.863935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:33.868029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 825/1639] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:33 2022] PUT /identity/v3/projects/77599bb8215e49e1b498591af0eaf977/users/107a1c486be94ca3a0b542341ab35921/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:33.868957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.893950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 815/1640] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:33 2022] PUT /identity/v3/projects/f6625f5c82d84fc49bc92ded91365c7c/users/33941ba8581e4385ae7032d61d77be58/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:33.895265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:33.967031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd48f9e-c449-417a-b554-d80ca826c7d6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:33.991533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:34.040617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c8293-1a71-4ee1-bdca-cdbffe0ab1a6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.048393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.049076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.049359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.049645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.052740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.054892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e2e01-29ad-4643-9f2b-0eb8e714348a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.057398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.059028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.059376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.059669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] PATH_INFO: `/v3/projects/ee3362848f30450f8cb418b07e70e35f/users/bbefbc57f5a34859a75bd147b211c06f/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.063966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3362848f30450f8cb418b07e70e35f, user_id=bbefbc57f5a34859a75bd147b211c06f, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.067361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4265b-855b-4232-b0cd-152f66e969ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.082740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 826/1641] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:33 2022] PUT /identity/v3/projects/ee3362848f30450f8cb418b07e70e35f/users/bbefbc57f5a34859a75bd147b211c06f/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:34.084988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.085300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.085584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.112118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 816/1642] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:33 2022] POST /identity/v3/projects => generated 403 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:34.112118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428b73-e1d4-4335-9634-eaa813467a51 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.112118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428b73-e1d4-4335-9634-eaa813467a51 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.112118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8428b73-e1d4-4335-9634-eaa813467a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.136328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:34.138800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:34.189720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a8428b73-e1d4-4335-9634-eaa813467a51 None None] MFA Rules not processed for user `107a1c486be94ca3a0b542341ab35921`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:34.191202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a8428b73-e1d4-4335-9634-eaa813467a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:34.228075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:34.269374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 827/1643] 10.222.0.24 () {58 vars in 1058 bytes} [Tue May 24 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:34.270928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.375647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 817/1644] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:34.380552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.386623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.387021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.423174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e7d32-c007-494d-ad5f-ac3a15ad948f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.440779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.443648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.445194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.445824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.453926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.458785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] MFA Rules not processed for user `bbefbc57f5a34859a75bd147b211c06f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:34.460411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:34.465029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.481293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 828/1645] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:34.482491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.515428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5c54a-bf80-4142-b244-27abe2641f08 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.528189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.528877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.529161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.529440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.530969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.536093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.578782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 829/1646] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:34 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:34.578782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.614809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce685b5-1a97-4735-93c9-9416c69867f7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.623113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 818/1647] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:34.624042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.629569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.631554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.631554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.631842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.681226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:34.749939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033ce0b-4d13-4729-b20f-8f00099a38ff None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.761869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.762483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.765379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.765379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.765760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.766543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.786608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.787594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 830/1648] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:34.789132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.803141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.817969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 819/1649] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:34 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:34.819091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.844894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7505ce-70e4-4279-928f-7d6b05eb71d9 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.857734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d64c-6e84-43a1-a9e6-12ec02117404 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:34.865505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.870214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.871804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:34.872345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:34.872751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.872880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] PATH_INFO: `/v3/projects/f6625f5c82d84fc49bc92ded91365c7c/users/33941ba8581e4385ae7032d61d77be58/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.873417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:34.874004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:34.879344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.882032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.884727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6625f5c82d84fc49bc92ded91365c7c, user_id=33941ba8581e4385ae7032d61d77be58, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:34.890791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:34.915230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 820/1650] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:34.920185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.931479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 831/1651] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:34 2022] PUT /identity/v3/projects/f6625f5c82d84fc49bc92ded91365c7c/users/33941ba8581e4385ae7032d61d77be58/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:34.931479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:34.962898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72771c-77f3-4730-8017-ca7bcd8b4b4a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.049468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:35.102715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.103591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.104026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.104457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.127715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86654053-f42f-44b2-a062-80ef728ad77e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.146608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.149869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.150220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.151043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] PATH_INFO: `/v3/projects/67edb58da3a04946994452faf9c9630b/users/4c9e1ef282814c38b4fc2ab1f35eefb7/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.197026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67edb58da3a04946994452faf9c9630b, user_id=4c9e1ef282814c38b4fc2ab1f35eefb7, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.201200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d19572-c09a-481a-b272-95a66b9f1ae5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.238752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.241054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.271878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 832/1652] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:34 2022] PUT /identity/v3/projects/67edb58da3a04946994452faf9c9630b/users/4c9e1ef282814c38b4fc2ab1f35eefb7/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:35.272092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.275002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.275320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.277817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 821/1653] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:35.278845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:35.302388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a056b-4f67-46be-936f-cf8fe9e82042 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.403579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] MFA Rules not processed for user `33941ba8581e4385ae7032d61d77be58`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:35.406457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:35.537565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.538265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.538558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.538849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.543725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.561340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0e0d8-8c96-4e4c-aff2-2d8da1935889 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.562914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 833/1654] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:35.563662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:35.592512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:35.600310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 822/1655] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:35.604033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb521-0e3e-441d-8ad3-b43747febbc6 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.604587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb521-0e3e-441d-8ad3-b43747febbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.604965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedb521-0e3e-441d-8ad3-b43747febbc6 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.606936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 823/1656] 10.222.0.24 () {54 vars in 912 bytes} [Tue May 24 13:55:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:35.608093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb521-0e3e-441d-8ad3-b43747febbc6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:35.647248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532a20-7520-4d09-a60a-74db7751318f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.652832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.653514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.653801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.654080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.704828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedb521-0e3e-441d-8ad3-b43747febbc6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.709868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.710626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.710981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.711436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.778367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.780051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.810744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 834/1657] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:35.812077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:35.831583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:35.859324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1910d718-7c70-4795-97be-a4e0e59b2938 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:35.876127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:35.877064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.877508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.877819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] PATH_INFO: `/v3/projects/67edb58da3a04946994452faf9c9630b/users/4c9e1ef282814c38b4fc2ab1f35eefb7/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.879503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.881734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ed748-f9e2-4f74-b567-78f9ed455b80 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.887105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=67edb58da3a04946994452faf9c9630b, user_id=4c9e1ef282814c38b4fc2ab1f35eefb7, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:35.893468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:35.914523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 824/1658] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:35 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:35.919171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:35.919696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:35.920120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:35.942933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 835/1659] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:35 2022] PUT /identity/v3/projects/67edb58da3a04946994452faf9c9630b/users/4c9e1ef282814c38b4fc2ab1f35eefb7/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:35.996798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:35.998568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:36.204269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 825/1660] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:55:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:36.216158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:36.216695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:36.219164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:36.358193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402cfd7-b425-4926-8f2e-0b5f96bcd131 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:36.361691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae49d-674b-48e0-8abf-7ae3d4e07d1f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:36.371229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:36.373832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:36.374331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:36.374871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:36.375313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:36.376014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:36.377150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:36.381596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:36.405199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:36.406806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:36.430939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 836/1661] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:36.517847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:36.519405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90eccb-8c5b-4355-9fd1-5fcbed73ca93 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:36.551278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 826/1662] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:36.826519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:36.980976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7500-f520-48ca-ac01-c807e906801d service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:37.003105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:37.006150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.006511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.006824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] PATH_INFO: `/v3/users/04bb2299091f45e8a53477aee4255e88` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:37.019120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bb2299091f45e8a53477aee4255e88)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:37.021196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:37.021196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 24 13:55:37.021196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: May 24 13:55:37.022672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:37.034669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:37.168688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bb2299091f45e8a53477aee4255e88'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:37.200311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bb2299091f45e8a53477aee4255e88'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:37.273108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bb2299091f45e8a53477aee4255e88'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:37.276629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd8ca-a514-448e-9925-2c9ab55dac54 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.277013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd8ca-a514-448e-9925-2c9ab55dac54 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.277537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd8ca-a514-448e-9925-2c9ab55dac54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:37.307401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bb2299091f45e8a53477aee4255e88'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:37.313814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b8f0795f-d506-4c16-8e6a-cfaddf335707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bb2299091f45e8a53477aee4255e88'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:37.365970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 837/1663] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:36 2022] DELETE /identity/v3/users/04bb2299091f45e8a53477aee4255e88 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:37.380224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.380581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.380912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:37.426213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-45ebd8ca-a514-448e-9925-2c9ab55dac54 None None] MFA Rules not processed for user `4c9e1ef282814c38b4fc2ab1f35eefb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:37.430422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-45ebd8ca-a514-448e-9925-2c9ab55dac54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:37.437948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:37.472236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:37.482846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:37.725589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 827/1664] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:37.727419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599a0e-0b79-4f2e-8797-70b4995dc5a5 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.727723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599a0e-0b79-4f2e-8797-70b4995dc5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.728361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599a0e-0b79-4f2e-8797-70b4995dc5a5 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:37.730928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 828/1665] 10.222.0.24 () {54 vars in 912 bytes} [Tue May 24 13:55:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:37.737811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff65a-8add-4c7a-9a65-fdddeb158649 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.740469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff65a-8add-4c7a-9a65-fdddeb158649 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.744259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff65a-8add-4c7a-9a65-fdddeb158649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:37.804255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 838/1666] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:37.845856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:37.847478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:37.914374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-442ff65a-8add-4c7a-9a65-fdddeb158649 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:37.917278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-442ff65a-8add-4c7a-9a65-fdddeb158649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:37.981810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2dca51-dd84-4586-843c-d239646016dd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:37.988148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:37.989023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:37.989141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:37.989389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:38.008688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:38.011288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:38.039924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 839/1667] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:38.087334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 829/1668] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:55:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:38.126288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:38.139498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f522e2-ddd0-48b3-9bf4-e5e439c28336 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:38.150482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:38.151416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:38.151884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:38.152321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:38.170648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:38.172410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3fa0b-8415-4dae-826a-e80f4504ccad service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:38.191041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 840/1669] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:39.715055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ca08-6f79-45a2-93f7-32006892097b None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:39.715961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ca08-6f79-45a2-93f7-32006892097b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:39.716390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d23ca08-6f79-45a2-93f7-32006892097b None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:39.718279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 830/1670] 10.222.0.24 () {54 vars in 960 bytes} [Tue May 24 13:55:39 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:39.724141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24715e-cefc-49c8-a8d5-4e5e6f96b3ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:39.725294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24715e-cefc-49c8-a8d5-4e5e6f96b3ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:39.725731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24715e-cefc-49c8-a8d5-4e5e6f96b3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:39.758438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0e24715e-cefc-49c8-a8d5-4e5e6f96b3ff None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:39.759964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e24715e-cefc-49c8-a8d5-4e5e6f96b3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:39.777652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:39.932067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 841/1671] 10.222.0.24 () {58 vars in 1057 bytes} [Tue May 24 13:55:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:39.940331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23ca08-6f79-45a2-93f7-32006892097b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:39.973009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d23ca08-6f79-45a2-93f7-32006892097b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:40.004919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:40.005612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.005897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.006580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.008697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:40.044221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:40.046430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:40.063107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 831/1672] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:40.281935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a401af-e987-42d0-9dff-693a9cd9ce16 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.282465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a401af-e987-42d0-9dff-693a9cd9ce16 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.283046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a401af-e987-42d0-9dff-693a9cd9ce16 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.286302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 842/1673] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:55:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:40.314052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:40.314772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:40.317656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.317881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.318653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.354782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37dbaec-22c0-4174-bc62-aa5ce4e7e128 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:40.371664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:40.372952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:40.392175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:40.392175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.393120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.393120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] PATH_INFO: `/v3/projects/002783353d9248ea837d793d9f5df662` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.400117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=002783353d9248ea837d793d9f5df662)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:40.405521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: WARNING py.warnings [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 13:55:40.405521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: warnings.warn(msg) May 24 13:55:40.405521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: May 24 13:55:40.409255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:40.441098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 843/1674] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:55:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:40.541414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:40.558201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733d810-98da-4a53-9c91-1541f1a35605 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:40.565472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '002783353d9248ea837d793d9f5df662', 'request_id': 'req-b34af25d-3ee8-4a4b-8421-cdf604502f27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:40.586350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '002783353d9248ea837d793d9f5df662', 'request_id': 'req-b34af25d-3ee8-4a4b-8421-cdf604502f27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:40.607436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '002783353d9248ea837d793d9f5df662', 'request_id': 'req-b34af25d-3ee8-4a4b-8421-cdf604502f27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:40.643340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] The token cache is being invalidate because project 002783353d9248ea837d793d9f5df662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:40.643594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:40.644330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:40.653624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 832/1675] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:40 2022] DELETE /identity/v3/projects/002783353d9248ea837d793d9f5df662 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:40.658117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:40.660599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:40.684871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:40.758025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:40.760209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.762812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.764253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.770834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:40.806306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:40.813515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:40.844156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 844/1676] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:40.890416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34af25d-3ee8-4a4b-8421-cdf604502f27 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:40.919720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:40.920680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:40.920974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:40.921248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] PATH_INFO: `/v3/users/33941ba8581e4385ae7032d61d77be58` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:40.928353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=33941ba8581e4385ae7032d61d77be58)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:40.930356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:41.086430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33941ba8581e4385ae7032d61d77be58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.089230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33941ba8581e4385ae7032d61d77be58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.149333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33941ba8581e4385ae7032d61d77be58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.151172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33941ba8581e4385ae7032d61d77be58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.160802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33941ba8581e4385ae7032d61d77be58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.185160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 833/1677] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:40 2022] DELETE /identity/v3/users/33941ba8581e4385ae7032d61d77be58 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:55:41.472007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:41.476928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:41.543541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afdf91-9354-496a-9748-85a29853f950 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:41.645798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:41.646709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:41.647329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:41.647329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] PATH_INFO: `/v3/users/2e5e815227ff47f485db253b984509f4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:41.649110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5e815227ff47f485db253b984509f4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:41.652009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:41.727747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:41.752145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5e815227ff47f485db253b984509f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.773166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5e815227ff47f485db253b984509f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.799593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5e815227ff47f485db253b984509f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.801202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5e815227ff47f485db253b984509f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.802085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e73d4771-10c3-4f62-9b0c-be44abbfda82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5e815227ff47f485db253b984509f4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:41.845698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 845/1678] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:41 2022] DELETE /identity/v3/users/2e5e815227ff47f485db253b984509f4 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:41.989004 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:41.990601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:42.016781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4875ea9-f88a-4ec7-a0ee-d8b55c6fd2af admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:42.148184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:42.148677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:42.149022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:42.149481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] PATH_INFO: `/v3/projects/f6625f5c82d84fc49bc92ded91365c7c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:42.151160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6625f5c82d84fc49bc92ded91365c7c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:42.153696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:42.236632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:42.277961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6625f5c82d84fc49bc92ded91365c7c', 'request_id': 'req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.288631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6625f5c82d84fc49bc92ded91365c7c', 'request_id': 'req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.298453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6625f5c82d84fc49bc92ded91365c7c', 'request_id': 'req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.320930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] The token cache is being invalidate because project f6625f5c82d84fc49bc92ded91365c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:42.321278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.322022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.328105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 834/1679] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:41 2022] DELETE /identity/v3/projects/f6625f5c82d84fc49bc92ded91365c7c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:42.330823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:42.331124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:42.331413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:42.361507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:42.362748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:42.389071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:42.390936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:42.588190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eeffce-03ab-4a35-8cc1-776bf7a9cec8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:42.601016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:42.602117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:42.602230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:42.602570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] PATH_INFO: `/v3/projects/7eda19672b7c450d8c3b6df5aa3abe21` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:42.603936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eda19672b7c450d8c3b6df5aa3abe21)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:42.607337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:42.735218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eda19672b7c450d8c3b6df5aa3abe21', 'request_id': 'req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.747904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eda19672b7c450d8c3b6df5aa3abe21', 'request_id': 'req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.759941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eda19672b7c450d8c3b6df5aa3abe21', 'request_id': 'req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.801169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] The token cache is being invalidate because project 7eda19672b7c450d8c3b6df5aa3abe21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:42.804139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.806006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-db0659fd-bad2-42f2-bb8a-1d2fb4a69bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:42.809263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 835/1680] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:42 2022] DELETE /identity/v3/projects/7eda19672b7c450d8c3b6df5aa3abe21 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:55:42.813814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:42.814244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:42.814657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:42.845066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 846/1681] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:42.847205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:42.849431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:42.864395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ac0624-d419-4f58-bda0-f8e09b973b47 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:42.892851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] MFA Rules not processed for user `3eb0459bdac1456a900046e264835a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:42.894226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:42.954938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:42.956365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:42.956955 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:42.957502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] PATH_INFO: `/v3/users/4c9e1ef282814c38b4fc2ab1f35eefb7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:42.959991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c9e1ef282814c38b4fc2ab1f35eefb7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:42.962555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:43.009507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:43.045484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c9e1ef282814c38b4fc2ab1f35eefb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.057026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:43.066297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c9e1ef282814c38b4fc2ab1f35eefb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.091455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c9e1ef282814c38b4fc2ab1f35eefb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.092268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c9e1ef282814c38b4fc2ab1f35eefb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.093005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5180f6e1-b0e6-4da5-bd51-70bf94313a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c9e1ef282814c38b4fc2ab1f35eefb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.120517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 847/1682] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:42 2022] DELETE /identity/v3/users/4c9e1ef282814c38b4fc2ab1f35eefb7 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:43.124273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:43.124681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:43.126179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:43.149085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 836/1683] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:43.164644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:43.167837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:43.178766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:43.189146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:43.305973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 848/1684] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:43.319008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:43.358874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76817eca-50b5-4b32-99bc-4fc095f2e9fd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:43.381767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:43.384454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:43.386729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:43.394332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:43.395603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:43.397286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:43.408773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:43.428359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71be4a-2a9b-427c-80e6-90b362509fce None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:43.464635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 849/1685] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:43.464635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:43.472562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:43.531749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7e310-ae5b-4d05-a09e-089325aefe57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:43.563339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:43.566994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:43.568069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:43.570173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] PATH_INFO: `/v3/users/7c0f2d7211af44a3a39e4da9613bebec` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:43.576340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0f2d7211af44a3a39e4da9613bebec)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:43.584930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:43.661170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:43.662322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:43.665520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:43.665980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:43.718072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0f2d7211af44a3a39e4da9613bebec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.749902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0f2d7211af44a3a39e4da9613bebec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.774299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0f2d7211af44a3a39e4da9613bebec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.780049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0f2d7211af44a3a39e4da9613bebec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.787850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0f2d7211af44a3a39e4da9613bebec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:43.830343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 850/1686] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:43 2022] DELETE /identity/v3/users/7c0f2d7211af44a3a39e4da9613bebec => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:43.831422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:43.836912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:43.855183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:43.859103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:43.888676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 837/1687] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:43.889941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:43.891730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:43.919151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00599ea-b6c2-426c-8636-62f0fd303ff5 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:43.941456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:43.961596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:44.054205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7e1f2-3559-41b3-bead-ae66d62d8eb3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:44.061237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.062681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.063198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.064409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.066111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.068629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b30cb-5963-495a-9353-d61a5f28f778 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:44.115809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.116739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.117027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.117744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] PATH_INFO: `/v3/projects/67edb58da3a04946994452faf9c9630b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.119030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67edb58da3a04946994452faf9c9630b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.122521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:44.224087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 851/1688] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:43 2022] POST /identity/v3/projects => generated 365 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:44.232945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3537910-e032-4d64-a6ba-710b8935d858 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.233224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3537910-e032-4d64-a6ba-710b8935d858 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.233922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3537910-e032-4d64-a6ba-710b8935d858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.241228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67edb58da3a04946994452faf9c9630b', 'request_id': 'req-2293dd0d-b630-4a6d-857c-a9707b4bfb54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:44.255999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67edb58da3a04946994452faf9c9630b', 'request_id': 'req-2293dd0d-b630-4a6d-857c-a9707b4bfb54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:44.273095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67edb58da3a04946994452faf9c9630b', 'request_id': 'req-2293dd0d-b630-4a6d-857c-a9707b4bfb54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:44.303329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] The token cache is being invalidate because project 67edb58da3a04946994452faf9c9630b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:44.303872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:44.306057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:44.307981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d3537910-e032-4d64-a6ba-710b8935d858 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:44.312775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d3537910-e032-4d64-a6ba-710b8935d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.314442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 838/1689] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:43 2022] DELETE /identity/v3/projects/67edb58da3a04946994452faf9c9630b => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:44.315629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:44.320715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.493395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 852/1690] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:44.493583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293dd0d-b630-4a6d-857c-a9707b4bfb54 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:44.493672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.493672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.493672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.518323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.519083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.519375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.519656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.521257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.523889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcd5ea-90f6-4651-a74d-ce72431fed2c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:44.557072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 839/1691] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:44 2022] POST /identity/v3/users => generated 283 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:44.562354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.562666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.563057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.564566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:44.566026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.596846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:44.598024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.625039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 840/1692] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:44.626149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:44.631946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.780746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:44.803555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 853/1693] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:44.804433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:44.833599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2894f-42f9-4e90-bcda-99c2b5bac1ec None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:44.837501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9fbc-65c6-4930-9ebd-35cc7d72bd97 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:44.849782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.851095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.852007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.852746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.853526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.853830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.856675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.856766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.856766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.860661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188224a6-6cf2-4d2b-b6ec-557cbb76e772 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:44.885361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.891111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:44.904082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 854/1694] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:44.904082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.904082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.904531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.940965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 841/1695] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:44.942425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:44.973304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85b15-e225-4f5e-8873-8338144ad5b5 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:44.988840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:44.989045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:44.989532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:44.989921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:44.990101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:44.991209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:44.993468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:44.998586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d335ca-304d-4b57-9a1b-e575a8773544 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:45.018563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 855/1696] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:45.020692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:45.043407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 842/1697] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:45.045056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.046962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fe20b-5838-4ab0-9979-8524740ce270 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:45.061628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:45.062474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.063123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.073434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.083289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.083709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.086586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:45.089449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:45.095683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:45.138588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] MFA Rules not processed for user `532b36f272dc432aad6faa294bd1bc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:45.140450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:45.153007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 856/1698] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:45.154584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:45.181825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96560-5ec3-43ce-81cd-1d2b22dbdf88 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:45.196412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:45.201805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.202056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.202345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] PATH_INFO: `/v3/projects/97f5e23e0db444088ec160de3c671619/users/cda62d7d26c341459eb9b53cf0a19409/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.241493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f5e23e0db444088ec160de3c671619, user_id=cda62d7d26c341459eb9b53cf0a19409, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:45.247481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:45.274479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 857/1699] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:45 2022] PUT /identity/v3/projects/97f5e23e0db444088ec160de3c671619/users/cda62d7d26c341459eb9b53cf0a19409/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:45.276917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:45.284979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:45.296584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:45.311204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 843/1700] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:45.312370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:45.348834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5e12f-7910-4a74-9988-307ea02b3cc2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:45.498524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27683d49-4dfe-4bdd-8a57-5d4f8a14a319 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:45.507066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:45.507066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.507066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.508026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.509685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:45.515510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f999b31-4955-43d4-8778-6a8da2364c76 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:45.523592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:45.524434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.525134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.528302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.610971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 844/1701] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:45 2022] POST /identity/v3/projects => generated 357 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:45.613132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.618292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.621051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.710200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] MFA Rules not processed for user `cda62d7d26c341459eb9b53cf0a19409`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:45.715751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:45.743015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:45.748318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb08eb-0b78-4a14-9c11-97429415ed0b service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:45.780170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 858/1702] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:55:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:55:45.785343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:45.789763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:45.789763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:45.793376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:45.889082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] MFA Rules not processed for user `532b36f272dc432aad6faa294bd1bc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:45.896091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:45.938884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 845/1703] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:45.940490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.108846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 859/1704] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:46.109835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.112324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.229016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd47a5d-5023-495d-b43a-ecb0f1278ed4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.239026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:46.239326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.240902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.241844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.242497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.243432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.246274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.269149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd8f8f-15e5-478f-aab4-40f0c29994ac None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.275026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.275989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.276284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.278232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] PATH_INFO: `/v3/projects/4fd016cc76ae4093afd1a12d340ddc65` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.278232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd016cc76ae4093afd1a12d340ddc65)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.282300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.317615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 846/1705] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:45 2022] POST /identity/v3/users => generated 279 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:46.318714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.344467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cdcdb-a3c0-48fc-8659-ccd9ed34dfbd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.366990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.367690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.368065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.368352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.369070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.371129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.377709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd016cc76ae4093afd1a12d340ddc65', 'request_id': 'req-e6b75c9a-e785-4708-b876-382c5c6305dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.386042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd016cc76ae4093afd1a12d340ddc65', 'request_id': 'req-e6b75c9a-e785-4708-b876-382c5c6305dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.394190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd016cc76ae4093afd1a12d340ddc65', 'request_id': 'req-e6b75c9a-e785-4708-b876-382c5c6305dc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.419463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] The token cache is being invalidate because project 4fd016cc76ae4093afd1a12d340ddc65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:46.419947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.420736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.422606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 860/1706] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:46 2022] DELETE /identity/v3/projects/4fd016cc76ae4093afd1a12d340ddc65 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:46.423192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.424656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.449599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 847/1707] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:46 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:46.453639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.455397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.528181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b75c9a-e785-4708-b876-382c5c6305dc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.615592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c949954-9078-4133-838a-521642b24d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.616323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c949954-9078-4133-838a-521642b24d9b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.616617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c949954-9078-4133-838a-521642b24d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.616898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c949954-9078-4133-838a-521642b24d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.620155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c949954-9078-4133-838a-521642b24d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.626536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd745b0-8776-4218-83ed-15a5f087b084 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.636998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:46.639717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.643254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.643992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.646111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] PATH_INFO: `/v3/users/2f00eaae5afb48808260a8836f5b4d27` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.649131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f00eaae5afb48808260a8836f5b4d27)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.655775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.691843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c949954-9078-4133-838a-521642b24d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.693657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c949954-9078-4133-838a-521642b24d9b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.705383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 861/1708] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:46.706571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c949954-9078-4133-838a-521642b24d9b service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.708431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c949954-9078-4133-838a-521642b24d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.732371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c949954-9078-4133-838a-521642b24d9b service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.745166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.745965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.746246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.746621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.752512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.755331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:46.761914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.785895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f00eaae5afb48808260a8836f5b4d27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.786876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f00eaae5afb48808260a8836f5b4d27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.807803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 862/1709] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:46.809378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.811499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.832018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f00eaae5afb48808260a8836f5b4d27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.832877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f00eaae5afb48808260a8836f5b4d27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.840626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3baca8-f5a2-4dfb-9a67-3bcecc415023 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:46.847563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f00eaae5afb48808260a8836f5b4d27'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:46.855843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:46.856881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:46.857243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:46.857794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] PATH_INFO: `/v3/users/bbefbc57f5a34859a75bd147b211c06f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:46.859485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbefbc57f5a34859a75bd147b211c06f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:46.861713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:46.871638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 848/1710] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:46 2022] DELETE /identity/v3/users/2f00eaae5afb48808260a8836f5b4d27 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:46.875659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:46.880762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:46.982481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbefbc57f5a34859a75bd147b211c06f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:47.016898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbefbc57f5a34859a75bd147b211c06f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:47.062103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbefbc57f5a34859a75bd147b211c06f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:47.065694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbefbc57f5a34859a75bd147b211c06f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:47.067684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbefbc57f5a34859a75bd147b211c06f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:47.167569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3599c7-5946-4073-a6f2-997df131f36c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.167780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 863/1711] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:46 2022] DELETE /identity/v3/users/bbefbc57f5a34859a75bd147b211c06f => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:47.167780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.173383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:47.318135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.319601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.320685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.321142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.322509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.326065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6caa19b-2483-4ab2-80b2-b5338bd91119 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.327894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.347808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.348515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.348835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.349120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.350232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.352102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce47a6-33ca-43d7-a65a-5d5d1015e63b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.377918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 864/1712] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:47.381561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.382888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.383342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.392252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 849/1713] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:46 2022] POST /identity/v3/users => generated 290 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:47.392252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.414033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b5924-4e71-4bde-98da-7cfd0fa59c4e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.418399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:47.419910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:47.437278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.439025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.439664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.440161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] PATH_INFO: `/v3/projects/04c991d8d6ed439dbe5d56bb7bf74796/users/9f4ab03b44a44519a41c1c93ee1d368e/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.447863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 865/1714] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:47.447863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.453634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:47.477994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c991d8d6ed439dbe5d56bb7bf74796, user_id=9f4ab03b44a44519a41c1c93ee1d368e, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.484018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.489142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7efa2b-3d6c-4730-8012-673145730a1b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.511010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.512102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.512907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.513309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.514343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.517181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e040490-1c3e-45d0-94e9-6e865f24914c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.519396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:47.533656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 850/1715] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:47 2022] PUT /identity/v3/projects/04c991d8d6ed439dbe5d56bb7bf74796/users/9f4ab03b44a44519a41c1c93ee1d368e/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:47.533656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.533656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:47.545046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 866/1716] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:47.548993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.549288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.551588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.589654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:47.617248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] MFA Rules not processed for user `9f4ab03b44a44519a41c1c93ee1d368e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:47.618497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:47.656190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a75257-73b4-41a3-99f5-ac764a1ead07 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.660926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.662849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.663270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.663669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.820606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.824480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.839735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 867/1717] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:47 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:47.840729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.858737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 851/1718] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:47.860062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.872508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709d239-e9ee-4767-84c9-3ed632f85b84 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.882007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.883038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.883220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.883474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] PATH_INFO: `/v3/projects/2f946d0ac47b41a88c45ee86ced871e2/users/0a97af2877cd4e6ca2a03a8981e1ab57/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.887587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaac4a-d191-4c39-b7b1-53f36ea74b5e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:47.905373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:47.906423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:47.907198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:47.907937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:47.908886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.912543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.926086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f946d0ac47b41a88c45ee86ced871e2, user_id=0a97af2877cd4e6ca2a03a8981e1ab57, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:47.929441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:47.959237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 868/1719] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:47 2022] PUT /identity/v3/projects/2f946d0ac47b41a88c45ee86ced871e2/users/0a97af2877cd4e6ca2a03a8981e1ab57/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:47.963372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:47.981786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 852/1720] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:47 2022] POST /identity/v3/projects => generated 355 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:47.986263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.097934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3feb4f3-292f-4c26-b69c-d08744a2499b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.106678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.107721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.108272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.108776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.109933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.112906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.137986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9e8ec-76ff-4c5c-b5a6-17b2cdd5298a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.155271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.156367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.156674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.156965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.157725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.159385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.167186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 853/1721] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:47 2022] POST /identity/v3/users => generated 284 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:48.174278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.218049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 869/1722] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:55:48.219662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.234036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea38eb-7942-479f-b8e6-10e933bd2278 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.242919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.244036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.245168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.245975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.246889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.248616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b070cd9-73d9-4185-b7f1-a853432a98c7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.249127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.263436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.265005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.265413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.265806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] PATH_INFO: `/v3/projects/2f946d0ac47b41a88c45ee86ced871e2/users/0a97af2877cd4e6ca2a03a8981e1ab57/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.267067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 854/1723] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:48.271402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.271704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f946d0ac47b41a88c45ee86ced871e2, user_id=0a97af2877cd4e6ca2a03a8981e1ab57, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.274619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a968b-02d4-4a0c-9656-c10147745d1f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.297134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f0990-6da6-4529-8e15-a1eb3b4a0521 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.320149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 870/1724] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:48 2022] PUT /identity/v3/projects/2f946d0ac47b41a88c45ee86ced871e2/users/0a97af2877cd4e6ca2a03a8981e1ab57/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:48.323290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.323708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.324128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.399587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:48.414183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] MFA Rules not processed for user `0a97af2877cd4e6ca2a03a8981e1ab57`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:48.418755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:48.505947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.509083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.509774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.510186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] PATH_INFO: `/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f/users/238bcaaa796c4535b8cf0a65197c33cc/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.568351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:48.575683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=363dd52da7ce4e6b9c39dce79fddb24f, user_id=238bcaaa796c4535b8cf0a65197c33cc, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.578723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.614274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 855/1725] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:48 2022] PUT /identity/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f/users/238bcaaa796c4535b8cf0a65197c33cc/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:55:48.617690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.668845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 871/1726] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:48 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:48.681316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.684011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:48.763339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79bf4e-f2b7-4b2b-86ad-491d3c46fc0f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.773139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.774620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.775170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.776103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.777843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.782236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.822291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 856/1727] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:48.826578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:48.848948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b661b4c-032e-4805-a99a-2685c309b39f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.855518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:48.857584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.858046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.859063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] PATH_INFO: `/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f/users/238bcaaa796c4535b8cf0a65197c33cc/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.859795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df4ac7-ba1e-4bb9-bcb0-78bb796aa5a8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:48.872295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363dd52da7ce4e6b9c39dce79fddb24f, user_id=238bcaaa796c4535b8cf0a65197c33cc, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:48.878624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f730d2-d03d-45ad-b43b-e4d74a540666 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:48.908517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 857/1728] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:48 2022] PUT /identity/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f/users/238bcaaa796c4535b8cf0a65197c33cc/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:48.920126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:48.920456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:48.920910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:48.989981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] MFA Rules not processed for user `238bcaaa796c4535b8cf0a65197c33cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:48.991733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.002069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:49.002762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:49.003161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:49.006922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:49.148369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 858/1729] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:48 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:55:49.150303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:49.151932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.194632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:49.257517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e7595-c0e1-42cb-9f9f-411e435cb267 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:49.275417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:49.279070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:49.279070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:49.279410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] PATH_INFO: `/v3/projects/ee3362848f30450f8cb418b07e70e35f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:49.282246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3362848f30450f8cb418b07e70e35f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:49.284703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:49.321220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:49.322960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:49.367209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 872/1730] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:48 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:49.367209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:49.386822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.399494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3362848f30450f8cb418b07e70e35f', 'request_id': 'req-e781ce49-cddb-4802-92ba-757dc27d6ee3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.408884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3362848f30450f8cb418b07e70e35f', 'request_id': 'req-e781ce49-cddb-4802-92ba-757dc27d6ee3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.418038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3362848f30450f8cb418b07e70e35f', 'request_id': 'req-e781ce49-cddb-4802-92ba-757dc27d6ee3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.451097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] The token cache is being invalidate because project ee3362848f30450f8cb418b07e70e35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:49.451405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.452723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.471264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 859/1731] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:49 2022] DELETE /identity/v3/projects/ee3362848f30450f8cb418b07e70e35f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:49.472412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:49.474576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.477038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690b66-b4e0-44b2-b181-43e515a78d0b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:49.677370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:49.681679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ce49-cddb-4802-92ba-757dc27d6ee3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:49.688103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:49.689999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:49.690771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:49.691558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:49.694339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.777251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:49.778160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:49.778479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:49.779069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] PATH_INFO: `/v3/projects/47cda95585534d33aebb09e4aadfb1fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:49.792923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47cda95585534d33aebb09e4aadfb1fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:49.801188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:49.847245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:49.860064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982ea80-0e58-4988-a1d5-2262d3b9477f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:49.866940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:49.883182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 860/1732] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:49 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:49.885620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057a82-3aa9-47f0-af73-2e22ee8864e2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:49.886591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057a82-3aa9-47f0-af73-2e22ee8864e2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:49.887176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057a82-3aa9-47f0-af73-2e22ee8864e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:49.915511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-72057a82-3aa9-47f0-af73-2e22ee8864e2 None None] MFA Rules not processed for user `532b36f272dc432aad6faa294bd1bc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:49.918470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-72057a82-3aa9-47f0-af73-2e22ee8864e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:49.968676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47cda95585534d33aebb09e4aadfb1fb', 'request_id': 'req-cc3043f7-1fd7-4452-bb36-94abb9baaa84'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:49.987258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47cda95585534d33aebb09e4aadfb1fb', 'request_id': 'req-cc3043f7-1fd7-4452-bb36-94abb9baaa84'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:50.009152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47cda95585534d33aebb09e4aadfb1fb', 'request_id': 'req-cc3043f7-1fd7-4452-bb36-94abb9baaa84'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:50.042538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] The token cache is being invalidate because project 47cda95585534d33aebb09e4aadfb1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:50.043452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:50.044575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc3043f7-1fd7-4452-bb36-94abb9baaa84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:50.047069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 873/1733] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:49 2022] DELETE /identity/v3/projects/47cda95585534d33aebb09e4aadfb1fb => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:50.050465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac894795-599e-4891-95ad-23c230e22910 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.050751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac894795-599e-4891-95ad-23c230e22910 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.051023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac894795-599e-4891-95ad-23c230e22910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.067082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 861/1734] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:50.070651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05123-af17-46e9-a422-65417434ff72 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.071274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05123-af17-46e9-a422-65417434ff72 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.071274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f05123-af17-46e9-a422-65417434ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.134309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ac894795-599e-4891-95ad-23c230e22910 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:50.140633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac894795-599e-4891-95ad-23c230e22910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.414263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 874/1735] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:50.415381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac894795-599e-4891-95ad-23c230e22910 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:50.417233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac894795-599e-4891-95ad-23c230e22910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.532651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac894795-599e-4891-95ad-23c230e22910 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:50.536906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:50.537608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.537925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.538233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.540504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.565497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:50.664102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:50.665293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bfe2-06c3-4af7-934d-67167f9fb353 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:50.670819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-52f05123-af17-46e9-a422-65417434ff72 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:50.675099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-52f05123-af17-46e9-a422-65417434ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.681090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 875/1736] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:50.683743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.684167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.684357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.706533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:50.711152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.776596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 876/1737] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:50.778111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:50.779198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.806254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 862/1738] 10.222.0.55 () {58 vars in 1015 bytes} [Tue May 24 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 739 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:50.809034 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b02-c13e-4bbe-88f1-71171cf647c8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.810073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b02-c13e-4bbe-88f1-71171cf647c8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.812610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb4b02-c13e-4bbe-88f1-71171cf647c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.816044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785970eb-275e-497d-b8de-62f2916eba33 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:50.831705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:50.834615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.834934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.835220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.842980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.853847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b1cb4b02-c13e-4bbe-88f1-71171cf647c8 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:50.857202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b1cb4b02-c13e-4bbe-88f1-71171cf647c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:50.887312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 863/1739] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:50.890919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16687-5aa3-4cee-a88f-70442b85732f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:50.891522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16687-5aa3-4cee-a88f-70442b85732f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:50.892173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced16687-5aa3-4cee-a88f-70442b85732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:50.935779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ced16687-5aa3-4cee-a88f-70442b85732f None None] MFA Rules not processed for user `3eb0459bdac1456a900046e264835a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:50.937427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ced16687-5aa3-4cee-a88f-70442b85732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:51.082311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.089794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.108899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 864/1740] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:51.109812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.110159 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.110649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.117165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 877/1741] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:51.120335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.122590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:51.136830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:51.138483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:51.155479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618293a-31dd-49f3-8be2-d6ed84f554b3 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.167284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 865/1742] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:51.169059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.171859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.172343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.172927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.179787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.208685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf14615-4abd-4285-b87a-922997558e2b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.213734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.218418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ffd49-f64f-4324-ba18-87426dbb3e6e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.223968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.227277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.228634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.229804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.232217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:51.240922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 878/1743] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:51.241452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.243885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ce62-e123-427b-9dd8-b3cf0c7b8577 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.250017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb70d67-9fc6-40d4-941c-c9e10dedd4b5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.253898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb70d67-9fc6-40d4-941c-c9e10dedd4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.254633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb70d67-9fc6-40d4-941c-c9e10dedd4b5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.261317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 879/1744] 10.222.0.24 () {54 vars in 912 bytes} [Tue May 24 13:55:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:51.266228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb70d67-9fc6-40d4-941c-c9e10dedd4b5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.275906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 866/1745] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:51.280334 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.280631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.280925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.329160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb70d67-9fc6-40d4-941c-c9e10dedd4b5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.353516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.358547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.359867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.360166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.361337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:51.364006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:51.367849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.368222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:51.372860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.399538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 867/1746] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:55:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:51.404161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.427425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 880/1747] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:51.429666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.442974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac150b71-dd5f-49ab-b11a-9d4c886a9a83 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.457212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.458136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.458776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0dea48-0929-4e3c-847c-89f00b45f27e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.459258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.459544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.461565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.469162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.471262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.472582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.474018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.474376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.476051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.481347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.555238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 868/1748] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:51 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:51.556454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.566777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 881/1749] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:51 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:51.567978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:51.568095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:51.823889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d2473-21bf-4fe3-b4a4-861aca609a31 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.832042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062c069-f521-43de-93f7-3d1adbe01e12 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:51.861875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:51.862710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:51.863045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:51.863331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:51.876524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:51.878901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:51.961542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:52.022471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.023166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.029275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.029646 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.042250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 882/1750] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:51 2022] POST /identity/v3/users => generated 285 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:52.043558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.062619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.064121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.095167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5f3b0-ce98-4570-a68d-55112276c243 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.103737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 869/1751] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:52.110727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.126773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.127565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.127876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.128161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.134428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.154052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.177254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c01ab-c2f8-489c-830f-3474e4b23cad service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.205758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.206607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.206805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.207093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.207938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.209561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.237684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 883/1752] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:52 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:52.240595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.274037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 870/1753] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:52.276496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.295355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea939ae-39a8-4277-a686-751ff0d871b4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.318418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0def64-3126-49ca-b077-917640720322 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.320163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.323089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.325729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.327655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.329762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.337447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.340649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.341346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.341642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.341933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.342687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.344935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.386485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 884/1754] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:52.387596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.393276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 871/1755] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:52.394293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.419345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b131-c98c-421d-8580-4df91c3bee0f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.419594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5562e-690b-40c0-b8ed-fb158f1962d7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.433242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.434171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.434529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.434793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] PATH_INFO: `/v3/projects/595f1013ac60424cbe6d2dcdb02158e0/users/9e38437a44874c3a93f15bfa664e668a/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.441158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.442098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.444271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.447956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.447956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.451971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.478190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595f1013ac60424cbe6d2dcdb02158e0, user_id=9e38437a44874c3a93f15bfa664e668a, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.481671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c94641-7fd8-4e17-bd10-30eded08aae6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.490292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 885/1756] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:52.506692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.527153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 872/1757] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:52 2022] PUT /identity/v3/projects/595f1013ac60424cbe6d2dcdb02158e0/users/9e38437a44874c3a93f15bfa664e668a/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:52.530086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecc558-ee43-4012-9099-35ac230b3d6d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.538096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.538398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.538692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.599038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] MFA Rules not processed for user `9e38437a44874c3a93f15bfa664e668a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:52.600380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:52.683660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:52.684748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.685106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.685463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] PATH_INFO: `/v3/projects/40eb072e6b8348e2bd1c58ce421983d2/users/c812cde8c08445ab8cde8d96a46f314c/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.715125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=40eb072e6b8348e2bd1c58ce421983d2, user_id=c812cde8c08445ab8cde8d96a46f314c, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:52.718839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca9280-00a0-40f5-ac0d-115becb4b73b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:52.755084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 886/1758] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:52 2022] PUT /identity/v3/projects/40eb072e6b8348e2bd1c58ce421983d2/users/c812cde8c08445ab8cde8d96a46f314c/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:52.782534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:52.786235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:52.788405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:52.799404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 873/1759] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:52.800801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:52.814719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:52.827296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63ad62-63b6-47ac-9e2e-bf32976caf00 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:52.918187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] MFA Rules not processed for user `c812cde8c08445ab8cde8d96a46f314c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:52.925256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:53.019540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:53.021554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:53.039299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:53.042678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:53.043333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:53.045828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:53.119971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 874/1760] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:52 2022] POST /identity/v3/projects => generated 369 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:55:53.130486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:53.162891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 887/1761] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:53.264861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42626698-7040-4166-87fe-0fa3a3c01fd2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:53.271767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:53.272747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:53.273044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:53.273361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:53.274340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:53.276435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:53.331738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 875/1762] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:53 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:53.333007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:53.347668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b076a01-4eff-480e-b37a-01cbbcf47685 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:53.355648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:53.356375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:53.356662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:53.356947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:53.357755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:53.359377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:53.423917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 888/1763] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:55:53.430170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:53.449825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:53.452093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:53.464660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06550362-ac75-4b96-98bc-14472f015485 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:53.515341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:53.517309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:53.518461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:53.532132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] PATH_INFO: `/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8/users/67c792c039af42ae8941c05c55f05fdb/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:53.572849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:53.605373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7e432875914c34bc9fb3f0b969c7f8, user_id=67c792c039af42ae8941c05c55f05fdb, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:53.615864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:53.663930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 876/1764] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:53 2022] PUT /identity/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8/users/67c792c039af42ae8941c05c55f05fdb/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:53.673299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:53.799271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d335688b-e2d2-4926-8e7e-0df1b76a9532 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:53.970018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:53.990740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910eb0b-75f8-4bec-8a69-752440b640c8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:54.005665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:54.006438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:54.006735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:54.008166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:54.008947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:54.015109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:54.015643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:54.016539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:54.016833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:54.017179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:54.050654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 877/1765] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:54.055250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:54.101528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e33b99-80fa-4012-b758-a176e92f1033 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:54.134414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:54.135905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:54.138207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6240e2d6-f2c1-4b34-a6e1-fab0a2f5ca89 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:54.161281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:54.162999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:54.163175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 889/1766] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:54.164180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] PATH_INFO: `/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8/users/67c792c039af42ae8941c05c55f05fdb/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:54.170113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7e432875914c34bc9fb3f0b969c7f8, user_id=67c792c039af42ae8941c05c55f05fdb, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:54.176100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:54.224087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 878/1767] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:54 2022] PUT /identity/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8/users/67c792c039af42ae8941c05c55f05fdb/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:54.240217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26aa4-696e-41b8-91c0-e3edc0b08ecf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:54.241391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26aa4-696e-41b8-91c0-e3edc0b08ecf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:54.243033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e26aa4-696e-41b8-91c0-e3edc0b08ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:54.382675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-40e26aa4-696e-41b8-91c0-e3edc0b08ecf None None] MFA Rules not processed for user `67c792c039af42ae8941c05c55f05fdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:54.401844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-40e26aa4-696e-41b8-91c0-e3edc0b08ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:54.530507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:54.570497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 890/1768] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:54.694237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:54.915505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77178b-a2bc-4a63-a162-bb2933dde7e6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:54.923886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:54.925313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:54.926778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:54.927117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:54.940277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:54.941871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b950-eb21-411b-8802-fb0c486e6ddc service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:54.956097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 879/1769] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:55:55.061984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:55.062798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:55.062798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:55.065209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 891/1770] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:55:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:55:55.085566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:55.086821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:55.087387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:55.156731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:55.158197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:55.264064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 880/1771] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:55.321525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:55.324505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:55.359145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4a2c2-327c-49df-a8d7-617e83924562 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:55.375665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:55.376717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:55.377196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:55.377631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:55.427446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:55.432749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968f414-fcee-4c25-9a7f-c176c53f0c50 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:55.471802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 892/1772] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:55.560458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:55.654546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a4984-570f-48a9-a88b-c5e4313b4bb0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:55.661757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:55.663166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:55.663460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:55.663742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] PATH_INFO: `/v3/users/c812cde8c08445ab8cde8d96a46f314c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:55.673902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c812cde8c08445ab8cde8d96a46f314c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:55.676516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:55.708506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:55.797331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c812cde8c08445ab8cde8d96a46f314c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:55.800154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c812cde8c08445ab8cde8d96a46f314c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:55.833904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c812cde8c08445ab8cde8d96a46f314c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:55.835798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c812cde8c08445ab8cde8d96a46f314c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:55.854076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c812cde8c08445ab8cde8d96a46f314c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:55.892831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 881/1773] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:55 2022] DELETE /identity/v3/users/c812cde8c08445ab8cde8d96a46f314c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:55.898117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:55.898660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:55.899170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:55.936480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:55.938475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:56.126865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 893/1774] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:56.138696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:56.141615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:56.346364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf6c4a-78a3-43f3-9e30-0c2a26d80b32 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:56.356067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:56.356624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:56.356838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:56.357148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:56.375759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:56.397304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583a414-877b-4d5c-accc-b053a55eb165 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:56.421505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 882/1775] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:57.727066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:57.730248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:57.761426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f174ed-f422-4369-abe2-b928961fe050 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:57.780288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:57.785360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:57.787022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:57.788367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] PATH_INFO: `/v3/users/0e2bd48d78a540d6978f128f9af94e0a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:57.793936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e2bd48d78a540d6978f128f9af94e0a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:57.802152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:57.948900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e2bd48d78a540d6978f128f9af94e0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:57.972010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e2bd48d78a540d6978f128f9af94e0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.047116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e2bd48d78a540d6978f128f9af94e0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.051246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e2bd48d78a540d6978f128f9af94e0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.052812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e2bd48d78a540d6978f128f9af94e0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.101533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 894/1776] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:57 2022] DELETE /identity/v3/users/0e2bd48d78a540d6978f128f9af94e0a => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:58.106894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:58.107506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:58.107506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:58.146060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:58.148008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.163135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:58.165421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.279074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:58.353065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713cfb8-5d8f-4d89-9bc6-a387768e624a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:58.369543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:58.370624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:58.370980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:58.371302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] PATH_INFO: `/v3/projects/40eb072e6b8348e2bd1c58ce421983d2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:58.374213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40eb072e6b8348e2bd1c58ce421983d2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:58.376564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:58.427928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 883/1777] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:58.482567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40eb072e6b8348e2bd1c58ce421983d2', 'request_id': 'req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.487465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:58.490953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.492480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40eb072e6b8348e2bd1c58ce421983d2', 'request_id': 'req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.509038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40eb072e6b8348e2bd1c58ce421983d2', 'request_id': 'req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.541067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] The token cache is being invalidate because project 40eb072e6b8348e2bd1c58ce421983d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:55:58.541415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.542989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ccd7d92d-d235-4306-9d16-14ca9bc7ad04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:58.546623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 895/1778] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:55:58 2022] DELETE /identity/v3/projects/40eb072e6b8348e2bd1c58ce421983d2 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:55:58.551712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64ba5-ec0e-46e2-9200-494acaa4a157 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:58.552094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64ba5-ec0e-46e2-9200-494acaa4a157 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:58.552427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf64ba5-ec0e-46e2-9200-494acaa4a157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:58.621042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cdf64ba5-ec0e-46e2-9200-494acaa4a157 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:58.623266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cdf64ba5-ec0e-46e2-9200-494acaa4a157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.638735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:58.696372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391f654-e9f1-4aa5-bca5-4496512918ec None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:58.865408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 896/1779] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:58.874511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:58.874905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:58.875269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:58.922874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:58.923652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:58.924039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:58.924416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:58.946327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.959476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:55:58.961039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:58.980817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 897/1780] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:55:58.982895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:58.984830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:58.985067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.003591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 884/1781] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:55:59.004991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528effe8-2c99-4970-8183-0320f26b6a77 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.031409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.033876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.034388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.048587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.051551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:55:59.059292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.061589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.092549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 898/1782] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:55:59.093846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.109780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928df88-47aa-4ebc-a84a-68823a9dfe8b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.126834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.128056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.128552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.129019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.130277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.132450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.182711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 885/1783] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:55:59 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:59.186979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.206289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892472de-b318-4375-b629-f7e7f70e1b28 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.351606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:55:59.387935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.389101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.389511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.389891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.405387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.407990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.479784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 899/1784] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:55:59 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:55:59.487632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.529241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb29521-59ab-40cc-8603-e428422b9d42 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.552321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.555776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.556128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.557333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.559601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.576910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.598828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.600834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:59.624823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 886/1785] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:59.629531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9f47c-9c87-42c4-8948-0b2dc1c55796 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.630537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.653358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.654743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.655698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.656999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] PATH_INFO: `/v3/users/9e38437a44874c3a93f15bfa664e668a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.659422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e38437a44874c3a93f15bfa664e668a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.663613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed173e1-b518-4f03-9e57-539fcdefda72 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.674428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.675509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.676239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.676531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.678424 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.682018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.697240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.749129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 887/1786] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:55:59.760977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.817686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e38437a44874c3a93f15bfa664e668a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:59.818124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0489dcb2-961e-4cc2-bcd7-85a0877d3249 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:55:59.827920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e38437a44874c3a93f15bfa664e668a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:59.843458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e38437a44874c3a93f15bfa664e668a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:59.844074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:55:59.845832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e38437a44874c3a93f15bfa664e668a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:59.846852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e38437a44874c3a93f15bfa664e668a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:55:59.847367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:55:59.847674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:55:59.847994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] PATH_INFO: `/v3/projects/ad95e0c83a9442b59b7bbde5ccf92674/users/1a4ab86ccb9a417aae0fbdae8a1bf239/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:55:59.891934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 900/1787] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:59 2022] DELETE /identity/v3/users/9e38437a44874c3a93f15bfa664e668a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:55:59.894964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:55:59.899809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:55:59.911132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad95e0c83a9442b59b7bbde5ccf92674, user_id=1a4ab86ccb9a417aae0fbdae8a1bf239, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:55:59.935074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d6677-ba33-4db9-b763-a1098d2c55b3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:55:59.995091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 888/1788] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:55:59 2022] PUT /identity/v3/projects/ad95e0c83a9442b59b7bbde5ccf92674/users/1a4ab86ccb9a417aae0fbdae8a1bf239/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:00.008005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628eee3-a8ed-4556-8b08-e0c26d4f415d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.008005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628eee3-a8ed-4556-8b08-e0c26d4f415d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.008005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b628eee3-a8ed-4556-8b08-e0c26d4f415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.008325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca272ff1-494f-433b-a57f-d46827c6a759 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:00.053191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b628eee3-a8ed-4556-8b08-e0c26d4f415d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:00.055313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b628eee3-a8ed-4556-8b08-e0c26d4f415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.164993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:00.196438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:00.197376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.197664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.197947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] PATH_INFO: `/v3/users/cda62d7d26c341459eb9b53cf0a19409` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.199545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda62d7d26c341459eb9b53cf0a19409)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:00.201528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:00.328969 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 889/1789] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:00.331136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.331656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.332183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.360738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda62d7d26c341459eb9b53cf0a19409'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:00.382371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda62d7d26c341459eb9b53cf0a19409'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:00.405978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] MFA Rules not processed for user `1a4ab86ccb9a417aae0fbdae8a1bf239`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:00.407373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.413123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda62d7d26c341459eb9b53cf0a19409'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:00.414251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda62d7d26c341459eb9b53cf0a19409'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:00.415698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61b20ebe-9407-415a-8bcf-61e5e2f94259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda62d7d26c341459eb9b53cf0a19409'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:00.444305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 901/1790] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:55:59 2022] DELETE /identity/v3/users/cda62d7d26c341459eb9b53cf0a19409 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:00.450041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.450543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.450986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.463264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:00.491412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:00.492982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.595298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 890/1791] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:00.595298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:00.595298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.679869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 902/1792] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:00.680963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:00.696552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.720707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b731a-7014-419b-a0fe-4f3ef9075c2f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:00.724371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b908f-8c25-4a4e-9771-96d66c6291bc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:00.732072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:00.733122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.733723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.734236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.735238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:00.737478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:00.802621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 903/1793] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:00 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:00.803762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:00.813734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.865415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:00.866174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:00.866567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:00.866807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:00.874243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:00.923046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:00.927511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:00.959383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 891/1794] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:00.960483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:00.989636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985bab0-cf42-495a-8d6b-6e383bc14bc8 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:00.997207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fad76-1001-4047-9f60-f7d93c81fb71 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.020753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.021450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.021800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.022083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.035983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:01.136974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.137710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.137932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.138222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.138939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.147665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.155964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.157844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.174924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 904/1795] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:01.195420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 892/1796] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:00 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:01.196859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.210568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6971d-d9fc-47e3-8892-4485f24416a5 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.225655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.226389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.226568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.226707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.226964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.227722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.229292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:01.236997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.244357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:01.260027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a094b8-97d7-48ee-a4cc-28139c8458c1 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.273439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.274411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.274764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.275647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] PATH_INFO: `/v3/projects/2ad3abc8cf774b279c6c68c27a1877d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.277286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 905/1797] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:01.281874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.283723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad3abc8cf774b279c6c68c27a1877d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.290087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.312651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c566-fddc-411f-b0f5-e5bb05216a41 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.330300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.331550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.332140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.332613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] PATH_INFO: `/v3/projects/2de42f5a1d404800b8464ed39db816c8/users/d88ef7c5b58a48038ebbf01692e867c2/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.370737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de42f5a1d404800b8464ed39db816c8, user_id=d88ef7c5b58a48038ebbf01692e867c2, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.374353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.405237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 906/1798] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:01 2022] PUT /identity/v3/projects/2de42f5a1d404800b8464ed39db816c8/users/d88ef7c5b58a48038ebbf01692e867c2/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:01.408977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.536625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad3abc8cf774b279c6c68c27a1877d3', 'request_id': 'req-77b0b16f-67d3-4743-9d4a-068a68436cdb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:01.566841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad3abc8cf774b279c6c68c27a1877d3', 'request_id': 'req-77b0b16f-67d3-4743-9d4a-068a68436cdb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:01.583519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad3abc8cf774b279c6c68c27a1877d3', 'request_id': 'req-77b0b16f-67d3-4743-9d4a-068a68436cdb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:01.595770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ddceb-6082-470f-91cb-6c14efb0156f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.613009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] The token cache is being invalidate because project 2ad3abc8cf774b279c6c68c27a1877d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:01.613739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:01.616275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:01.620326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 893/1799] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:01 2022] DELETE /identity/v3/projects/2ad3abc8cf774b279c6c68c27a1877d3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:01.626744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.628861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:01.782476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.791006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.794408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.794699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.800078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.807636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.844321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0b16f-67d3-4743-9d4a-068a68436cdb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.848626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 907/1800] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:01.857038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:01.870224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:01.872794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.876022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.876375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.876695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] PATH_INFO: `/v3/users/107a1c486be94ca3a0b542341ab35921` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.881633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=107a1c486be94ca3a0b542341ab35921)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.883581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.911735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805f41fd-04f2-4ada-8f2e-9a17956392ed admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:01.951564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:01.953410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:01.953934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:01.955316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] PATH_INFO: `/v3/projects/2de42f5a1d404800b8464ed39db816c8/users/d88ef7c5b58a48038ebbf01692e867c2/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:01.964463 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de42f5a1d404800b8464ed39db816c8, user_id=d88ef7c5b58a48038ebbf01692e867c2, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:01.970363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb97e4f-4f8e-421f-bae5-c727b1ea8739 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:01.981342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:02.008516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 908/1801] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:01 2022] PUT /identity/v3/projects/2de42f5a1d404800b8464ed39db816c8/users/d88ef7c5b58a48038ebbf01692e867c2/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:02.014079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:02.014364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:02.016828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:02.019301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '107a1c486be94ca3a0b542341ab35921'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:02.020642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '107a1c486be94ca3a0b542341ab35921'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:02.053083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '107a1c486be94ca3a0b542341ab35921'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:02.056017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '107a1c486be94ca3a0b542341ab35921'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:02.083368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '107a1c486be94ca3a0b542341ab35921'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:02.090161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] MFA Rules not processed for user `d88ef7c5b58a48038ebbf01692e867c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:02.092181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:02.111663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 894/1802] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:01 2022] DELETE /identity/v3/users/107a1c486be94ca3a0b542341ab35921 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:02.197035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 909/1803] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:02.208901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:02.211184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:02.231854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:02.330252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f703985-ce80-4443-bf74-4858646a1da9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:02.342406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:02.343165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:02.343545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:02.343920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:02.461276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:02.463019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:02.476158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 895/1804] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:02.610310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:02.819210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c95c717-1dde-4ba2-bba8-3e8fab002cde None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:02.827187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:02.828131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:02.828612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:02.829076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:02.832130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:02.909327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:02.911030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:02.927457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 910/1805] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:03.064420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.065870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.148589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.150233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.273302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5b1047-da9f-42d5-8bc9-dc320269ed74 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.283440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:03.284749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.285261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.285825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] PATH_INFO: `/v3/projects/97f5e23e0db444088ec160de3c671619` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.289055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f5e23e0db444088ec160de3c671619)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:03.289640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:03.294380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:03.364128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc1c0a-72b2-4968-b747-43bc9050f13a service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.379959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:03.381846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.382213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.382562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] PATH_INFO: `/v3/projects/595f1013ac60424cbe6d2dcdb02158e0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.385310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=595f1013ac60424cbe6d2dcdb02158e0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:03.387877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:03.404169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f5e23e0db444088ec160de3c671619', 'request_id': 'req-0f7f204b-f42d-49b4-b5ca-05d3083dba88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.417442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f5e23e0db444088ec160de3c671619', 'request_id': 'req-0f7f204b-f42d-49b4-b5ca-05d3083dba88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.432515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:03.438736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f5e23e0db444088ec160de3c671619', 'request_id': 'req-0f7f204b-f42d-49b4-b5ca-05d3083dba88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.475362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] The token cache is being invalidate because project 97f5e23e0db444088ec160de3c671619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:03.475728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595f1013ac60424cbe6d2dcdb02158e0', 'request_id': 'req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.476166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.477864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-0f7f204b-f42d-49b4-b5ca-05d3083dba88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.484609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 896/1806] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:03 2022] DELETE /identity/v3/projects/97f5e23e0db444088ec160de3c671619 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:03.486383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.486868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.488104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.496310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 897/1807] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:56:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:56:03.497261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.499946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595f1013ac60424cbe6d2dcdb02158e0', 'request_id': 'req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.505003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.511237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595f1013ac60424cbe6d2dcdb02158e0', 'request_id': 'req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.534913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] The token cache is being invalidate because project 595f1013ac60424cbe6d2dcdb02158e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:03.536310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.538287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e3ee5c21-6f65-458c-8b48-db7a9e7ba8aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.540163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 911/1808] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:03 2022] DELETE /identity/v3/projects/595f1013ac60424cbe6d2dcdb02158e0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:03.546436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.546851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.547281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.597439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:03.598710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.622582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fbe9de-0ce4-489c-a575-2ebe37f42454 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.627256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:03.628813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.630020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.630352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] PATH_INFO: `/v3/users/0a97af2877cd4e6ca2a03a8981e1ab57` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.631595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a97af2877cd4e6ca2a03a8981e1ab57)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:03.633605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:03.685924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 912/1809] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:03.687017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.688540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.692310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a97af2877cd4e6ca2a03a8981e1ab57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.693191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a97af2877cd4e6ca2a03a8981e1ab57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.702994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac67976d-7902-4785-a3c2-ff0e4f597dad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.707557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:03.708467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.708605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a97af2877cd4e6ca2a03a8981e1ab57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.708758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.709044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] PATH_INFO: `/v3/users/67c792c039af42ae8941c05c55f05fdb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.709262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a97af2877cd4e6ca2a03a8981e1ab57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.710395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c792c039af42ae8941c05c55f05fdb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:03.712349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:03.716709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a97af2877cd4e6ca2a03a8981e1ab57'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.730952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 898/1810] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:03 2022] DELETE /identity/v3/users/0a97af2877cd4e6ca2a03a8981e1ab57 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:03.734704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.736724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.774104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c792c039af42ae8941c05c55f05fdb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.780894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:03.783412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c792c039af42ae8941c05c55f05fdb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.803690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c792c039af42ae8941c05c55f05fdb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.804586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c792c039af42ae8941c05c55f05fdb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.805218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c792c039af42ae8941c05c55f05fdb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.823319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b651f-38d5-4ebb-bae6-91da01a97911 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.830687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:03.831672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.832116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.832541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] PATH_INFO: `/v3/projects/77599bb8215e49e1b498591af0eaf977` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:03.833958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77599bb8215e49e1b498591af0eaf977)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:03.836305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:03.848107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 913/1811] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:03 2022] DELETE /identity/v3/users/67c792c039af42ae8941c05c55f05fdb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:03.849501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:03.854386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:03.858133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:03.926814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77599bb8215e49e1b498591af0eaf977', 'request_id': 'req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.936660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77599bb8215e49e1b498591af0eaf977', 'request_id': 'req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.944817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77599bb8215e49e1b498591af0eaf977', 'request_id': 'req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.953675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f78708-17e2-42fc-9bd1-40b3b41531ca admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:03.970649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] The token cache is being invalidate because project 77599bb8215e49e1b498591af0eaf977 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:03.971556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.973264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b23b666c-7d2a-4b46-88fd-ce2d65f0aa97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:03.977336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 899/1812] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:03 2022] DELETE /identity/v3/projects/77599bb8215e49e1b498591af0eaf977 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:03.982586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176a94-80da-4385-afa6-fcb91ddeaae6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:03.983421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176a94-80da-4385-afa6-fcb91ddeaae6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:03.983904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06176a94-80da-4385-afa6-fcb91ddeaae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.006317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-06176a94-80da-4385-afa6-fcb91ddeaae6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:04.007560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-06176a94-80da-4385-afa6-fcb91ddeaae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:04.095889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.096736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.097261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.098400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.103293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:04.128787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 900/1813] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:04.129585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.131229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.131603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.132028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.136276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.162093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 914/1814] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:04.170179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:04.171680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:04.188138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 901/1815] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:04.189464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.204957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c63d2-29c8-41e1-81fc-48c26cf8ccde service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.210400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.211372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.211651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.212058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.213055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:04.213437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.215249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.234034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 915/1816] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:04.235187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.246722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772c0b3-7c7d-4985-8b52-fdd2f326674a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.253130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.253312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.253536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.253819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.254888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.257017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.301992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 902/1817] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:04 2022] POST /identity/v3/projects => generated 393 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:04.308650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.324878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d15df4-9c74-4b9d-9b9c-f1e9a1f82af2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.470750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:04.525590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.525793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.526205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.526497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.527210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.529202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.566621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 916/1818] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:04 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:04.570408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.583331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9dc3e-e2cd-495b-8ddf-8879ec6e2794 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.589656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.590346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.590635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.590917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.591767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.593801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.620517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 903/1819] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:04.624276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.649382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b18d1-1228-41cc-bdbf-544f10c73c73 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.658942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.660195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.660616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.661194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.662456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.664174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58051827-2e96-40e5-88d7-4c2365d76e8e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.680213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 917/1820] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:04.685251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:04.703150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26712-c872-4228-8ee3-8e92b7d5308c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:04.719594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:04.722795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.723022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.723321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] PATH_INFO: `/v3/projects/6104991490c14f308de99e5bb94252f0/users/40c0a165ff9647e0a3ecca7eb617221f/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.745774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6104991490c14f308de99e5bb94252f0, user_id=40c0a165ff9647e0a3ecca7eb617221f, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:04.748858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:04.780029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 904/1821] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:04 2022] PUT /identity/v3/projects/6104991490c14f308de99e5bb94252f0/users/40c0a165ff9647e0a3ecca7eb617221f/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:04.784701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:04.785298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:04.786064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:04.828648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] MFA Rules not processed for user `40c0a165ff9647e0a3ecca7eb617221f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:04.830448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:04.939889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 918/1822] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:04.945244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.023079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:05.055581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.057949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.077869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a510353-0b6d-4425-ab0d-bdbd06f8462b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.081966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8accf0-8043-411c-a5e0-04e3937978fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.087511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.088458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.089378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.089679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] PATH_INFO: `/v3/projects/2f946d0ac47b41a88c45ee86ced871e2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.090763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.091074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f946d0ac47b41a88c45ee86ced871e2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.091868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.092385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.093247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.093816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.095664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.099063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.164591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 905/1823] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:04 2022] POST /identity/v3/projects => generated 391 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:05.169499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.199451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f946d0ac47b41a88c45ee86ced871e2', 'request_id': 'req-89aea297-b21f-4ed0-b9db-41134c5d8452'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.208119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f946d0ac47b41a88c45ee86ced871e2', 'request_id': 'req-89aea297-b21f-4ed0-b9db-41134c5d8452'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.218704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f946d0ac47b41a88c45ee86ced871e2', 'request_id': 'req-89aea297-b21f-4ed0-b9db-41134c5d8452'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.237878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] The token cache is being invalidate because project 2f946d0ac47b41a88c45ee86ced871e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:05.238419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.239413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.241684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 919/1824] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:05 2022] DELETE /identity/v3/projects/2f946d0ac47b41a88c45ee86ced871e2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:05.242938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.245708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.260423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c3ede-0b56-484f-bf56-ad4e6fcf8a99 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.299006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:05.339175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.339942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.340245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.340974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.341675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.344206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aea297-b21f-4ed0-b9db-41134c5d8452 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.344937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18f2f2-00da-497f-b013-da0ed4fa2049 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.351648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.353875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.354233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.354588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] PATH_INFO: `/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.356173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7e432875914c34bc9fb3f0b969c7f8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.358221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.380082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 906/1825] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/users => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:05.381522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.381866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.382326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.403673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:05.405281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.414781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 907/1826] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:05.416038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.417568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.435265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a4df7-c7ac-454c-97cb-d18aa3a6d99e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.435854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7e432875914c34bc9fb3f0b969c7f8', 'request_id': 'req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.448636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7e432875914c34bc9fb3f0b969c7f8', 'request_id': 'req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.456879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7e432875914c34bc9fb3f0b969c7f8', 'request_id': 'req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.475595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] The token cache is being invalidate because project ce7e432875914c34bc9fb3f0b969c7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:05.476309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.476822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7f38d35d-7549-4e7e-a3ec-46e2ba614aa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:05.479651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 920/1827] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:05 2022] DELETE /identity/v3/projects/ce7e432875914c34bc9fb3f0b969c7f8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:05.481347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc5b5-ed41-4f4c-94c0-5743e60d8845 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.485527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc5b5-ed41-4f4c-94c0-5743e60d8845 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.485909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fc5b5-ed41-4f4c-94c0-5743e60d8845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.499246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:05.514171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c40fc5b5-ed41-4f4c-94c0-5743e60d8845 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:05.516061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c40fc5b5-ed41-4f4c-94c0-5743e60d8845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.526252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.527261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.527261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.528042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.528457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.533687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.536202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 921/1828] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:05.537665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.537973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.538239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.548940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 908/1829] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:05.550519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.566186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f23f8-0035-4a1c-997f-d484515413cf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.572340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.573668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.574037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.574398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.575275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:05.575663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.577475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.589005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:05.596817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 909/1830] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:05.596817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.597203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.604605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.619843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6495e-48c0-4566-995e-1e6bb17136e4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.630539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 922/1831] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:05.631532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.640265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.641187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.641483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.641771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] PATH_INFO: `/v3/projects/1d77bc721251496bada0b29f628cd190/users/47be788f56c547869eb6d70d1941b00d/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.657271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589b020-28d0-4256-919b-1e4e9e5596cc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.662760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.663442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.663731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.664029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d77bc721251496bada0b29f628cd190, user_id=47be788f56c547869eb6d70d1941b00d, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.664134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.664737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.667254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.667773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64028d8e-8c7a-4b51-8aa6-6c1bce6b50a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.711168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 923/1832] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:05.713500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.713804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.714089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.719384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 910/1833] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:05 2022] PUT /identity/v3/projects/1d77bc721251496bada0b29f628cd190/users/47be788f56c547869eb6d70d1941b00d/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:05.719384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.746425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:05.751672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:05.827810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bca02bf-2796-4a18-8158-23b175346e2a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.864503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 924/1834] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:05.865348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.875169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd43d00-ae09-4510-8a0d-5fdcb5c52357 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.880336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.881015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.881304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.881714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.882460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.884101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.896617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.901725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 925/1835] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:05.904100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.904443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.904733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.905472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.907308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.909146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.921727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c97da-ac6e-4d21-88c6-61f408ec156c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.927679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.929634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.930105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.930396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.931266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:05.931585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.933314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.934562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 911/1836] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:05.935511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.950617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 926/1837] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:05.950617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.950885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6401-b77b-4914-bebf-dd61083380cf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.954908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.955851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.956135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.956416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] PATH_INFO: `/v3/projects/1d77bc721251496bada0b29f628cd190/users/47be788f56c547869eb6d70d1941b00d/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.958442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d77bc721251496bada0b29f628cd190, user_id=47be788f56c547869eb6d70d1941b00d, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.959031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:05.961306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.964162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4810c07-a0bc-4a25-812b-b0bc43a0f510 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:05.972015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:05.972803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.973091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.973730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:05.974682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:05.976241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 912/1838] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:05 2022] PUT /identity/v3/projects/1d77bc721251496bada0b29f628cd190/users/47be788f56c547869eb6d70d1941b00d/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:05.976761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21fa77-4cb0-4a67-a894-526663ed9f01 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:05.980655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:05.993918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 927/1839] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:05.995396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:05.995689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:05.996003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.032155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:06.032364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] MFA Rules not processed for user `47be788f56c547869eb6d70d1941b00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:06.033550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:06.071764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f879bc-c99c-4faf-b38d-3195400ee50e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.078604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.079358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.080269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.080468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.081197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.084337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.110365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 928/1840] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:06.111241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.123667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d106-04ff-449b-b3b6-3599629b9af6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.128261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 913/1841] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:05 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:06.131360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.135130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:06.204958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bb604-085e-4f37-9917-d14e803c1bdc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.209193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.209853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.210168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.210451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.211293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.212143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.212586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.213133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.213957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.216033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.227681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 929/1842] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:06.229675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.240098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6be8d-c006-473b-9e2c-d9e1fbf9360c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.249184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.249901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.250190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.250470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.251184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.253581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.281392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 930/1843] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/users => generated 294 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:06.281392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.287883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.288937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b79407-853b-4c89-a8f4-71736ab80699 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.289500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.294630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.295526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.296021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.296300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] PATH_INFO: `/v3/projects/04c9a6ccc7e44dd08f4d944f45a61ae6/users/03e441c8a93a4ae2a94c5cb13a4743cf/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.302353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 914/1844] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:06.303354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.316616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb7f774-6676-416a-9436-f991bad6d776 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.322687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=04c9a6ccc7e44dd08f4d944f45a61ae6, user_id=03e441c8a93a4ae2a94c5cb13a4743cf, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.324228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.325291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.325404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.325660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.326607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.328554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.328882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5cca9-25f4-4560-8529-03ca1750c9ca admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.342211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 915/1845] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:06.346781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.349234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 931/1846] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:06 2022] PUT /identity/v3/projects/04c9a6ccc7e44dd08f4d944f45a61ae6/users/03e441c8a93a4ae2a94c5cb13a4743cf/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:06.362224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.362973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f8034-29a1-4a41-971f-53da1f33cc4f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.363085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.363550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.402695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] MFA Rules not processed for user `03e441c8a93a4ae2a94c5cb13a4743cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:06.403996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:06.433869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:06.434396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:06.450730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.451515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.451846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.452201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.453003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.454697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.470166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 916/1847] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:06.470166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.505804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 932/1848] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:06.506723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.523004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e567575-467f-411f-9d5c-a79a011db2b8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.529040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.529964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.530258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.530549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] PATH_INFO: `/v3/projects/e7f832ed8df1476ba5cb25c6c3a8a9d8/users/7421995b1ded4163ab6501e6215f0c47/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.538803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53c55c5-ebef-44c9-ab3e-c860b2b47bdd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.543394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.544426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.544875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.545166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.546589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7f832ed8df1476ba5cb25c6c3a8a9d8, user_id=7421995b1ded4163ab6501e6215f0c47, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.547023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:06.549754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.564759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 933/1849] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:06 2022] PUT /identity/v3/projects/e7f832ed8df1476ba5cb25c6c3a8a9d8/users/7421995b1ded4163ab6501e6215f0c47/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:06.565634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.608822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.610262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c91dc-0606-40e0-a8fe-7b296478183a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.619380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 917/1850] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:06.620753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.621045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.621332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.632653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4146b8b-5fdd-46ce-a98d-1b362ef1de47 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.638697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.639891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.640410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.641111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.642166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.644352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.651849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] MFA Rules not processed for user `7421995b1ded4163ab6501e6215f0c47`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:06.653439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:06.692574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 934/1851] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:06.693702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.740507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 918/1852] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:06.745221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.758290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7cbfb-23f4-4fb4-b0b5-6f1c202d5159 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.786848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257478d-964d-4ad9-87d0-12a63d9c73d5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.794278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.794749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.795327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.795450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.796182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.797177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:06.798269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.822242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 935/1853] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:06.824986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.835166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d423a-ae19-455d-9272-84e9c91e89e9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.840793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.841495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.841786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.842063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.842811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.844436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.846913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.848839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.849237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.850077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.850941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.855400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.859016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 936/1854] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:06.867628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.873656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:06.889409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21902fcc-cd61-425a-8485-cf2ef3eddd61 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:06.908271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:06.909220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:06.909513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:06.914880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] PATH_INFO: `/v3/projects/084682327eec40d7b2c3c60bcd510942/users/a75c789958a04542be471f9298bbd3b0/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:06.917548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 919/1855] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:06.918359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:06.949116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=084682327eec40d7b2c3c60bcd510942, user_id=a75c789958a04542be471f9298bbd3b0, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:06.954451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:06.983281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 937/1856] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:06 2022] PUT /identity/v3/projects/084682327eec40d7b2c3c60bcd510942/users/a75c789958a04542be471f9298bbd3b0/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:06.984226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.060130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea2fd2-270d-4720-84ea-8a122b1040a4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.123254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5f90d-783c-452b-aaae-c2559e0cf60a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.129554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.130326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.131106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.131395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.132564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.135329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.152624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 938/1857] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:07.156453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.160938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.161632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.161957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.162206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.162939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.164891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f568641-93cf-4a90-87bc-5d0587d304e8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.170034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348a3af-0460-4d3c-bba9-f2c6c444ec3c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.177609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.178703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.179095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.179532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] PATH_INFO: `/v3/projects/084682327eec40d7b2c3c60bcd510942/users/a75c789958a04542be471f9298bbd3b0/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.183520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=084682327eec40d7b2c3c60bcd510942, user_id=a75c789958a04542be471f9298bbd3b0, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.186466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.200321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 939/1858] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:07 2022] PUT /identity/v3/projects/084682327eec40d7b2c3c60bcd510942/users/a75c789958a04542be471f9298bbd3b0/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:07.203972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 920/1859] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:06 2022] POST /identity/v3/users => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:07.206218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.211045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.211517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.211787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.242592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] MFA Rules not processed for user `a75c789958a04542be471f9298bbd3b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:07.244065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:07.288423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce8868-1816-4342-84b8-b4619be17d5a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.298900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.299591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.299913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.300186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.300919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.303036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.316562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:07.327251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 940/1860] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:07.331267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.343256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc2bdd-f21f-4d57-803f-da4c580c08d8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.348289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.349673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 921/1861] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:07.350117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.350547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.350835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] PATH_INFO: `/v3/projects/240d9ce6e05d45cc907e7812f73527c2/users/32877b26fe8c403e9d2b93c0fc5da7e5/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.365236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.385962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=240d9ce6e05d45cc907e7812f73527c2, user_id=32877b26fe8c403e9d2b93c0fc5da7e5, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.385962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.410798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 941/1862] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:07 2022] PUT /identity/v3/projects/240d9ce6e05d45cc907e7812f73527c2/users/32877b26fe8c403e9d2b93c0fc5da7e5/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:07.414497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.426549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cdbde0-ad73-49ec-a87c-0ae02caa1c1b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.452065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:07.476716 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.477377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.477705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.478057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.481856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d901298-b097-4e68-8ae5-d388c6563f3f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.486421 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.487084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.487369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.487653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.488412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.490004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.503556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 942/1863] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:07.505070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.515287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39bfc0-1fcc-4246-bb59-51b25a821515 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.519851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.520732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.521019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.521294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] PATH_INFO: `/v3/projects/240d9ce6e05d45cc907e7812f73527c2/users/32877b26fe8c403e9d2b93c0fc5da7e5/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.523307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=240d9ce6e05d45cc907e7812f73527c2, user_id=32877b26fe8c403e9d2b93c0fc5da7e5, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.526116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e40f5e-c7a4-41af-a37f-eaa0103bd743 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.538336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 943/1864] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:07 2022] PUT /identity/v3/projects/240d9ce6e05d45cc907e7812f73527c2/users/32877b26fe8c403e9d2b93c0fc5da7e5/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:07.549542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.549915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.550129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.551207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.552843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.563333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 922/1865] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:07.578005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] MFA Rules not processed for user `32877b26fe8c403e9d2b93c0fc5da7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:07.579251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:07.650831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 944/1866] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:07.662257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.672039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7567c6df-1ab5-458e-b465-f223ff5ece59 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.723301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.723983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.724273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.724555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.732333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:07.734731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.736647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77da3e-c30d-4c0d-97c3-bc919222f71d service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.747046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 923/1867] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:07.814459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:07.891369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f5949-77b8-4aa3-a44e-0489552e1104 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:07.896285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:07.897277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:07.897638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:07.897991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] PATH_INFO: `/v3/users/7421995b1ded4163ab6501e6215f0c47` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:07.899278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7421995b1ded4163ab6501e6215f0c47)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:07.901967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:07.961183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:07.961183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7421995b1ded4163ab6501e6215f0c47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:07.968842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7421995b1ded4163ab6501e6215f0c47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:07.982725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7421995b1ded4163ab6501e6215f0c47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:07.983495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7421995b1ded4163ab6501e6215f0c47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:07.984267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7421995b1ded4163ab6501e6215f0c47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:07.999899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 945/1868] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:07 2022] DELETE /identity/v3/users/7421995b1ded4163ab6501e6215f0c47 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:08.003467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:08.003798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:08.004302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:08.022800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:08.024018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:08.084446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 924/1869] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:08.094600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:08.096040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:08.151783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27e1e9-2553-4b56-a563-8f2a3a917449 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:08.155407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:08.156085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:08.156379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:08.156658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:08.165530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:08.166995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:08.176540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 946/1870] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:56:08.454760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:08.458040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:08.479001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9bc110-3308-4393-ba8e-e731c92c4bd0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:08.482327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:08.483220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:08.483507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:08.483787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] PATH_INFO: `/v3/projects/e7f832ed8df1476ba5cb25c6c3a8a9d8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:08.485189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7f832ed8df1476ba5cb25c6c3a8a9d8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:08.487099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:08.554867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7f832ed8df1476ba5cb25c6c3a8a9d8', 'request_id': 'req-44d2fefc-9998-4b4e-98f5-ff4a94a026db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.569742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7f832ed8df1476ba5cb25c6c3a8a9d8', 'request_id': 'req-44d2fefc-9998-4b4e-98f5-ff4a94a026db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.578588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7f832ed8df1476ba5cb25c6c3a8a9d8', 'request_id': 'req-44d2fefc-9998-4b4e-98f5-ff4a94a026db'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.592404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] The token cache is being invalidate because project e7f832ed8df1476ba5cb25c6c3a8a9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:08.592779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.593554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.595116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 925/1871] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:08 2022] DELETE /identity/v3/projects/e7f832ed8df1476ba5cb25c6c3a8a9d8 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:08.599059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:08.609565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bddd7-80da-4a62-b9c0-7b119d9f3617 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:08.669584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:08.670401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:08.670686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:08.670970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] PATH_INFO: `/v3/users/32877b26fe8c403e9d2b93c0fc5da7e5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:08.672214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32877b26fe8c403e9d2b93c0fc5da7e5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:08.674162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:08.723747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32877b26fe8c403e9d2b93c0fc5da7e5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.731081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32877b26fe8c403e9d2b93c0fc5da7e5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.738174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:08.745388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32877b26fe8c403e9d2b93c0fc5da7e5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.746319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32877b26fe8c403e9d2b93c0fc5da7e5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.746975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6c3899c5-0b43-425a-b98e-1c18fc0e6375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32877b26fe8c403e9d2b93c0fc5da7e5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:08.760106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 947/1872] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:08 2022] DELETE /identity/v3/users/32877b26fe8c403e9d2b93c0fc5da7e5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:09.031625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.033080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:09.046483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d2fefc-9998-4b4e-98f5-ff4a94a026db admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.107836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.108709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.108998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.109278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] PATH_INFO: `/v3/projects/240d9ce6e05d45cc907e7812f73527c2` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.110451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240d9ce6e05d45cc907e7812f73527c2)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.112366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.135065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:09.159482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240d9ce6e05d45cc907e7812f73527c2', 'request_id': 'req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:09.166158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240d9ce6e05d45cc907e7812f73527c2', 'request_id': 'req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:09.171613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240d9ce6e05d45cc907e7812f73527c2', 'request_id': 'req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:09.183786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] The token cache is being invalidate because project 240d9ce6e05d45cc907e7812f73527c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:09.184101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:09.184742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1a6bc8f0-dfa9-4b82-a781-cf8bb62b1cc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:09.186313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 926/1873] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:09 2022] DELETE /identity/v3/projects/240d9ce6e05d45cc907e7812f73527c2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:09.191126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.191423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.191706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.209993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:09.211135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:09.277427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 948/1874] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:09.284955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.285253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.285570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.303946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:09.305441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:09.317083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 927/1875] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:09.321992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.331579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d24946-35fd-4f7a-a3c2-81a2a304a53d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.336398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.337748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.338118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.338335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.339096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:09.339411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.341052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.353260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 949/1876] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:09.357388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.367856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538afb94-79b9-4af5-834d-c8ee57e4fcf5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.374583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.375255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.375543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.376032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.376759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.378842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.406386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 928/1877] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:09.410509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.421073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e7ba5-7d98-49bc-8252-4c3a2f50e6f3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.482814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.483537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.483840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.484134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.484854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.486782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.514715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 950/1878] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:09.518310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.529032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba5adf-e9c3-411f-8ef3-4b1b2cad185a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.533659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.534324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.534619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.534904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.535636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.537265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.549512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 929/1879] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:09.553395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.564415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d1b90-92e9-44e8-b6d5-2bbfcd1dfd4a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.569143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.569806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.570098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.570381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.571117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.572825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358ee5c-83a3-4440-81e6-eb6d8bfdd5f7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.584792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 951/1880] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:09.589570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.601595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5369d29-1aa6-4319-aa8a-02ecaaed5810 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.609878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.610768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.611058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.611341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] PATH_INFO: `/v3/projects/950c39810a9d4e2083391886dac44016/users/043616485a8543caa442320b69cd23d3/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.627911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=950c39810a9d4e2083391886dac44016, user_id=043616485a8543caa442320b69cd23d3, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.631216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.646565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 930/1881] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:09 2022] PUT /identity/v3/projects/950c39810a9d4e2083391886dac44016/users/043616485a8543caa442320b69cd23d3/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:09.658661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.659203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.659435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.691259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] MFA Rules not processed for user `043616485a8543caa442320b69cd23d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:09.692826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:09.737318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:09.770594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 952/1882] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:09.775950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.842580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f3a99-897a-4cf1-9036-1954cdd8b324 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.847244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.847937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.848226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.848505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.849205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.851092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.879296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 931/1883] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:09.883555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.894237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96fc61-5624-45d6-8053-ead5ef921f06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.950182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.950861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.951150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.951433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.952167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.954084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:09.974163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 953/1884] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:09 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:09.978362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:09.988426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c1468e-5cab-413a-867c-cfbaf2746612 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:09.992983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:09.993725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:09.994065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:09.994351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:09.995116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:09.996748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.007297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 932/1885] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:10.011613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.021406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e6b97-2189-47c0-82a4-17382aff464e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.026939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.027601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.027919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.028197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.028925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.030511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f954a1-1cf9-42af-92cf-fe3bca24fb1b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.040718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 954/1886] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:10.044902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.055014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cbd20-117b-4488-8649-f10cc6c9106e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.060337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.061224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.061548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.061812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] PATH_INFO: `/v3/projects/8b88bb3e32c24be3a62b1ff05a0182cb/users/1ed9594a669647129e1972939afe9f6c/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.077013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b88bb3e32c24be3a62b1ff05a0182cb, user_id=1ed9594a669647129e1972939afe9f6c, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.079633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.082735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:10.095242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 933/1887] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:10 2022] PUT /identity/v3/projects/8b88bb3e32c24be3a62b1ff05a0182cb/users/1ed9594a669647129e1972939afe9f6c/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:10.105747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.105747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.106121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.132585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] MFA Rules not processed for user `1ed9594a669647129e1972939afe9f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:10.134262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:10.198446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 955/1888] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:10.203053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.263321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e6d43-4ef5-4bab-a10f-c31956a29ff7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.267998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.268662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.268952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.269253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.269958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.271909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.300959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 934/1889] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:10 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:10.302119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.315704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605c3c0-b547-479d-8d61-d9e42a9cfbd4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.358903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:10.373316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.374002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.374271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.374549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.375247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.377192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.400878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 956/1890] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:10 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:10.404958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.416441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69123432-aac3-40b8-96df-0e9400da12d5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.422410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.423090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.423498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.423786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.424557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.426147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.440362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 935/1891] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:10.444752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.454914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118907c-8daa-4425-8fb8-db56eee2a2a2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.459548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.460473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.460757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.461033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] PATH_INFO: `/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e/users/4e3e2ba37c534ded8382a19a318180b6/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.477174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8c45e07a604465b0db6b1ac4439d0e, user_id=4e3e2ba37c534ded8382a19a318180b6, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.480496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.498450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 957/1892] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:10 2022] PUT /identity/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e/users/4e3e2ba37c534ded8382a19a318180b6/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:10.505093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.515786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d41d90-093d-47b8-b6a6-2ae6a8ae3993 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.588136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.589560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.589560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.589903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.590840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.592913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363803-5a3b-4ef4-9a0d-ecdc71d920fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.608489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 936/1893] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:10.609636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.623268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7656e1-7bb0-45a1-8497-cc4f3799410c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.630544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.631704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.632083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.632460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] PATH_INFO: `/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e/users/4e3e2ba37c534ded8382a19a318180b6/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.635116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e8c45e07a604465b0db6b1ac4439d0e, user_id=4e3e2ba37c534ded8382a19a318180b6, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.638119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.655937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 958/1894] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:10 2022] PUT /identity/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e/users/4e3e2ba37c534ded8382a19a318180b6/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:10.664867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.665213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.665543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.704861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] MFA Rules not processed for user `4e3e2ba37c534ded8382a19a318180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:10.706078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:10.756213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:10.775838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 937/1895] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:10.786773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:10.788715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:10.853183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783032-3b9e-40a6-bff8-144f5cb57753 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:10.857153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:10.857818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:10.858108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:10.858909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:10.869190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:10.870788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:10.881869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 959/1896] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:11.511258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:11.520613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa98ae4-37d3-4c9d-8aeb-e77d244387cb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:11.524309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:11.524995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:11.525378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:11.525581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:11.586617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:11.588454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57115513-f45c-4612-9cbd-e2fa25c0ffd2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:11.597795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 938/1897] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:13.262889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:13.264909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:13.357146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910e5f7-972f-4c77-8257-2e47b7004888 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:13.362953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:13.364969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.365204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.365956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] PATH_INFO: `/v3/users/9f4ab03b44a44519a41c1c93ee1d368e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.367706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4ab03b44a44519a41c1c93ee1d368e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:13.371187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:13.440046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:13.443663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4ab03b44a44519a41c1c93ee1d368e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:13.451329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4ab03b44a44519a41c1c93ee1d368e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:13.468206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4ab03b44a44519a41c1c93ee1d368e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:13.469129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4ab03b44a44519a41c1c93ee1d368e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:13.469924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-78373c29-3f56-4e2e-bd44-6f363b2a3951 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4ab03b44a44519a41c1c93ee1d368e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:13.485242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 960/1898] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:13 2022] DELETE /identity/v3/users/9f4ab03b44a44519a41c1c93ee1d368e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:13.490007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.490460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.490730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.517717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:13.518931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:13.569333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea5651-aeb7-40eb-833d-50dca5fc1ef3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.569731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea5651-aeb7-40eb-833d-50dca5fc1ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.570477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea5651-aeb7-40eb-833d-50dca5fc1ef3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.571638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 961/1899] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:56:13.580027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.580515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.580937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.607770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:13.609343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:13.619287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 939/1900] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:13.633090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:13.635341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:13.718113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 962/1901] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:56:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:13.732678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:13.734207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:13.734764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88c0e3-0f83-4324-bb92-950eebb3b4d1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:13.739504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:13.740201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.740489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.740764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.750126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6681500-f78a-4af5-88f4-857bdce0dda6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:13.754508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:13.756573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:13.757481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:13.758274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:13.758755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:13.759246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:13.770111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:13.772047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:13.774050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 940/1902] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:13.785393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 963/1903] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:14.120200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:14.123200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:14.181793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cdb2a1-e19f-4af3-a99b-c7b42977efcd service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:14.200483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:14.203130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:14.203130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:14.203496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] PATH_INFO: `/v3/projects/04c991d8d6ed439dbe5d56bb7bf74796` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:14.209321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c991d8d6ed439dbe5d56bb7bf74796)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:14.211267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:14.284133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:14.320022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c991d8d6ed439dbe5d56bb7bf74796', 'request_id': 'req-938b5db5-d659-4b01-a92c-556970843af2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.333234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c991d8d6ed439dbe5d56bb7bf74796', 'request_id': 'req-938b5db5-d659-4b01-a92c-556970843af2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.349232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c991d8d6ed439dbe5d56bb7bf74796', 'request_id': 'req-938b5db5-d659-4b01-a92c-556970843af2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.366421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] The token cache is being invalidate because project 04c991d8d6ed439dbe5d56bb7bf74796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:14.367077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.368322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.370595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 941/1904] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:14 2022] DELETE /identity/v3/projects/04c991d8d6ed439dbe5d56bb7bf74796 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:14.374778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:14.392698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce7bf-75c7-4fb0-adcb-6088aacee033 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:14.491857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:14.492961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:14.493293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:14.494070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] PATH_INFO: `/v3/users/238bcaaa796c4535b8cf0a65197c33cc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:14.495691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=238bcaaa796c4535b8cf0a65197c33cc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:14.499288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:14.600344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:14.616157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '238bcaaa796c4535b8cf0a65197c33cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.643382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '238bcaaa796c4535b8cf0a65197c33cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.663206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '238bcaaa796c4535b8cf0a65197c33cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.664169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '238bcaaa796c4535b8cf0a65197c33cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.667216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b0ddd1dc-d8e7-4299-93c3-37e86a031798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '238bcaaa796c4535b8cf0a65197c33cc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:14.690556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 964/1905] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:14 2022] DELETE /identity/v3/users/238bcaaa796c4535b8cf0a65197c33cc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:14.823610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:14.826073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:14.849626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b5db5-d659-4b01-a92c-556970843af2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:14.980681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:14.981710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:14.982003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:14.982285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] PATH_INFO: `/v3/users/40c0a165ff9647e0a3ecca7eb617221f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:14.984190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40c0a165ff9647e0a3ecca7eb617221f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:14.986180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:15.043068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40c0a165ff9647e0a3ecca7eb617221f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.044096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40c0a165ff9647e0a3ecca7eb617221f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.077934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40c0a165ff9647e0a3ecca7eb617221f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.080588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40c0a165ff9647e0a3ecca7eb617221f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.094565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40c0a165ff9647e0a3ecca7eb617221f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.114836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 942/1906] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:14 2022] DELETE /identity/v3/users/40c0a165ff9647e0a3ecca7eb617221f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:15.119926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.120228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.120513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:15.139951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:15.141518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.235938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 965/1907] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:15.278165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:15.279721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.353879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:15.355144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.373303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:15.404292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c9267-8180-4f49-bb80-00130770529a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:15.409893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:15.410664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.411032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.411403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:15.423235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:15.424740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:15.438079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 943/1908] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:15.491367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a2244-1289-4609-bb9e-d14738bccc2d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:15.504426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:15.505738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.506126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.506618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:15.511706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.617165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:15.619503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e3fef0-44d7-4769-9ef8-28224fec7478 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:15.636256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 966/1909] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:15.653089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:15.655656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.680900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494352e2-3624-47c6-8fd5-d4b908a1fea8 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:15.699931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:15.702041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.704862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.705288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] PATH_INFO: `/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:15.709729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] RBAC: Authorizing `identity:delete_project(project_id=363dd52da7ce4e6b9c39dce79fddb24f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:15.715667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:15.794185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363dd52da7ce4e6b9c39dce79fddb24f', 'request_id': 'req-d766efaf-b193-4ce0-bbe6-d3b07516565a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.801329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363dd52da7ce4e6b9c39dce79fddb24f', 'request_id': 'req-d766efaf-b193-4ce0-bbe6-d3b07516565a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.808821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363dd52da7ce4e6b9c39dce79fddb24f', 'request_id': 'req-d766efaf-b193-4ce0-bbe6-d3b07516565a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.825631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] The token cache is being invalidate because project 363dd52da7ce4e6b9c39dce79fddb24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:15.826682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.827576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d766efaf-b193-4ce0-bbe6-d3b07516565a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:15.829503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 944/1910] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:15 2022] DELETE /identity/v3/projects/363dd52da7ce4e6b9c39dce79fddb24f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:15.835252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.835760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.837473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:15.863206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:15.865043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:15.876420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:15.980034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 967/1911] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:15.990765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:15.991305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:15.991833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.017315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:16.018827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.030080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 945/1912] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:16.034608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.047729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316071a1-d21a-4e03-bc28-cafdd55503c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.059152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.061723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.063120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.063803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.080263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:16.088956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.091036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.105883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.107110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.130027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 968/1913] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:16.134968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.136356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e6042-a978-4ae0-8a24-9ce808af08f3 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.145790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566dc0fc-465b-4434-beaf-5f22f119d19b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.150914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.151545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.151754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.152125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.152935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.156216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.159494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.160674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.161185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.161450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] PATH_INFO: `/v3/projects/6104991490c14f308de99e5bb94252f0` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.162938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6104991490c14f308de99e5bb94252f0)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.170887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.225302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 969/1914] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:16 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:16.226894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.303987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6104991490c14f308de99e5bb94252f0', 'request_id': 'req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.317964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6104991490c14f308de99e5bb94252f0', 'request_id': 'req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.327291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8da38-6b84-4e75-887c-7a18cbe8b9e7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.330699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6104991490c14f308de99e5bb94252f0', 'request_id': 'req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.352630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] The token cache is being invalidate because project 6104991490c14f308de99e5bb94252f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:16.352762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.353683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.358434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 946/1915] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:16 2022] DELETE /identity/v3/projects/6104991490c14f308de99e5bb94252f0 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:16.358434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.362215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.406371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.407869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.408348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.408809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.409762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.411982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.428525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:16.455198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3b5a1-00ab-46cd-ae65-a0e90002bac3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.459420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.460110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.460460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.460673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.463431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.464966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 970/1916] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:16 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:16.465712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.473260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.493924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8dc0f-d3d3-4829-a1ff-8fa64ccbd45b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.503471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.504790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.505293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.505680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] PATH_INFO: `/v3/users/47be788f56c547869eb6d70d1941b00d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.507995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47be788f56c547869eb6d70d1941b00d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.510432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.517626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:16.542850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.544333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.554697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 947/1917] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:16.556948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.561804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.582995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77abe782-bb45-45ba-9cd6-8c47712ad5a8 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.591798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.592805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.593578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47be788f56c547869eb6d70d1941b00d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.594168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.594667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.595770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.598053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.603963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47be788f56c547869eb6d70d1941b00d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.609472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 948/1918] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:16.613946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.624100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47be788f56c547869eb6d70d1941b00d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.625115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47be788f56c547869eb6d70d1941b00d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.625917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47be788f56c547869eb6d70d1941b00d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:16.626560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be12f3b-c66d-478e-bfb9-98d30e8082b5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.635423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.636162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.636462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.636752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.637618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.639516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403bfff8-77dd-4c64-85bd-df4cd0bcffbd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.647405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 971/1919] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:16 2022] DELETE /identity/v3/users/47be788f56c547869eb6d70d1941b00d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:16.654441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 949/1920] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:16.659902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:16.661873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:16.845509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd44a9-6f52-4c55-aba8-fcccc5e35d21 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:16.859832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:16.861098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.861411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.861702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] PATH_INFO: `/v3/projects/8ec0b997f16e47938eacaabe46031951/users/e5687ac509b244448aaa64736a0a060a/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.909783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec0b997f16e47938eacaabe46031951, user_id=e5687ac509b244448aaa64736a0a060a, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:16.914352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:16.944773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 972/1921] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:16 2022] PUT /identity/v3/projects/8ec0b997f16e47938eacaabe46031951/users/e5687ac509b244448aaa64736a0a060a/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:16.957256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:16.957902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:16.958030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:16.989878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] MFA Rules not processed for user `e5687ac509b244448aaa64736a0a060a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:16.991228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:17.076306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 950/1922] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:17.081804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.097929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.100015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:17.193706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f8c83-4371-458b-aa03-36770cf63fc0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.199597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.200288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.200575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.200861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.201576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.203629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.215932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:17.220496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51307a-dd05-4e3e-ac22-04aa462a74e5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.227246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.228795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.229252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.229762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] PATH_INFO: `/v3/projects/1d77bc721251496bada0b29f628cd190` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.231453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d77bc721251496bada0b29f628cd190)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.233576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.235684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:17.240102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 973/1923] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:17.245147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.329666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d77bc721251496bada0b29f628cd190', 'request_id': 'req-413fbf73-8b52-48cd-8e3d-33711782ea0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:17.339140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52707c5a-d6b2-49a0-8bf6-ae33095c4666 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.342544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d77bc721251496bada0b29f628cd190', 'request_id': 'req-413fbf73-8b52-48cd-8e3d-33711782ea0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:17.349677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d77bc721251496bada0b29f628cd190', 'request_id': 'req-413fbf73-8b52-48cd-8e3d-33711782ea0b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:17.372623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] The token cache is being invalidate because project 1d77bc721251496bada0b29f628cd190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:17.372923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:17.375332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-413fbf73-8b52-48cd-8e3d-33711782ea0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:17.378450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 951/1924] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:17 2022] DELETE /identity/v3/projects/1d77bc721251496bada0b29f628cd190 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:17.384145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.384442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.384824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.406212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:17.407537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:17.423583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.424353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.425046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.425246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.426708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.429171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.459253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 974/1925] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:17.462309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.466063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:17.487236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e8fd1-0630-4329-9d64-eef5373fb1ad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.503306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.504488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.505002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.505295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.506054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.507748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91fba4d-d0c0-419b-81e5-3596a7b3a3dc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.532191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 975/1926] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:17.533704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 952/1927] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:17.544526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.552228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.552572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.552843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.561861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba9b636-7279-474f-a3d0-6eb05b9a7449 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.574302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.575329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.575811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.576353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] PATH_INFO: `/v3/projects/13f40c41ac634858a22f0106fa23e59b/users/7952718961314833b9e0fd4e0523ab5e/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.621185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:17.623262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:17.637215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f40c41ac634858a22f0106fa23e59b, user_id=7952718961314833b9e0fd4e0523ab5e, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.642199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.646089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 976/1928] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:17.647364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.673574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5b4da-604a-432d-9128-225f6f57944c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.683389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 953/1929] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:17 2022] PUT /identity/v3/projects/13f40c41ac634858a22f0106fa23e59b/users/7952718961314833b9e0fd4e0523ab5e/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:17.688174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.792088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991252d-f292-4f53-9d14-1e46d4500c57 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.799146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.799972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.801007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.801394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.802142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.812778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.814897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.816165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.816778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.818080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.819856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 954/1930] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:17.823340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:17.823340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.824877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.826353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.838046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:17.849794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb093da1-12df-447d-a882-a1a4e17ff5fb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.854797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 977/1931] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:17.855884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.860251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.861269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.861566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.861971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] PATH_INFO: `/v3/projects/13f40c41ac634858a22f0106fa23e59b/users/7952718961314833b9e0fd4e0523ab5e/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.862678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:17.864318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f40c41ac634858a22f0106fa23e59b, user_id=7952718961314833b9e0fd4e0523ab5e, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.868667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6de685-7137-4583-8054-ac136068bffe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.870003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf15af7-8c94-4d61-a385-822e1c7a019a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:17.879746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:17.881384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.882935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.883049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 955/1932] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:17 2022] PUT /identity/v3/projects/13f40c41ac634858a22f0106fa23e59b/users/7952718961314833b9e0fd4e0523ab5e/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:17.884739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.886373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:17.888914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:17.897793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:17.898140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:17.898477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:17.925212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 978/1933] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:17.930860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:17.937756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] MFA Rules not processed for user `7952718961314833b9e0fd4e0523ab5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:17.940046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.021121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b3099-a991-41a0-9f98-65c788653e57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.027357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.028091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.028384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.028697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.029464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.031413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.040408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 956/1934] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:18.049243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.050619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.054440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 979/1935] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:17 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:18.059987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.070628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804d63b-4714-45ce-9c4b-9a58a7054a67 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.077057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.077740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.078033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.078316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.079064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.080838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.096403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 980/1936] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:18.097589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.107424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa934d-1dce-439d-8a42-8f0a8751ebf8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.113092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.113776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.114072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.114385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.115130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.116783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.117026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab3336-1691-498f-990e-efd4665cf4d0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.122092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.122751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.123038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.123315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.132899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 981/1937] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:18.134619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.135940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.137389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.145296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072282a-3ba7-489d-9fa8-255558ec8654 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.147557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 957/1938] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:56:18.150644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.151686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.152019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.152307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] PATH_INFO: `/v3/projects/a3bd6c91b0bd48df886b7ee6fec0833c/users/28bede01326c4e5993162ddbbd0790fc/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.169278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3bd6c91b0bd48df886b7ee6fec0833c, user_id=28bede01326c4e5993162ddbbd0790fc, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.172060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be545c1-9046-4576-b187-995eed7392ea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.178571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.187727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 982/1939] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:18 2022] PUT /identity/v3/projects/a3bd6c91b0bd48df886b7ee6fec0833c/users/28bede01326c4e5993162ddbbd0790fc/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:18.191380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4462cd-2382-4f2c-b9d3-d87999978e14 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.199795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.200136 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.200425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.245459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] MFA Rules not processed for user `28bede01326c4e5993162ddbbd0790fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:18.246897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.262670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.263682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.264064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.264352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] PATH_INFO: `/v3/users/e5687ac509b244448aaa64736a0a060a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.265605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5687ac509b244448aaa64736a0a060a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.267886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.296678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:18.322553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 983/1940] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:18.327009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.335514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5687ac509b244448aaa64736a0a060a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:18.336231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5687ac509b244448aaa64736a0a060a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:18.338672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81aa86a-0c9c-441e-a3fd-a8ee47271649 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.344747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.345659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.346050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.346437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.347306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.349380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.350316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5687ac509b244448aaa64736a0a060a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:18.351098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5687ac509b244448aaa64736a0a060a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:18.361537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cc163d6c-a7c8-49c3-9500-e3f8b8cf5ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5687ac509b244448aaa64736a0a060a'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:18.366102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:18.377626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 958/1941] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:18 2022] DELETE /identity/v3/users/e5687ac509b244448aaa64736a0a060a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:18.386520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 984/1942] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:18 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:18.386701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.386701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.386701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.387521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.389164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.420423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:18.421879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.464158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05e68c3-4fde-467b-a379-d36f697a45a1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.469291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.469976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.470268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.470540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.471350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.473371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.491098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 959/1943] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:18.499929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 985/1944] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:18 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:18.502727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.503361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.505409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.513949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997be91-0a2b-47a0-9abf-4a8d6fcd2a6a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.519340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.520039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.520324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.520604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.521339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.522963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.534027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 960/1945] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:18.538192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.548472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740f51a-45d8-4e2b-9fb4-76a780162e88 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.553299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.554174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.554466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.554743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] PATH_INFO: `/v3/projects/08a939739112446394c9bb8ff615fefd/users/b18ea04c50b047979b5d77babb60f84b/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.565813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9276a8-096c-41e8-8012-c8b221560152 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.569895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.570623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.570945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.571232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.571748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a939739112446394c9bb8ff615fefd, user_id=b18ea04c50b047979b5d77babb60f84b, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.574437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.581135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.582944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.589947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 961/1946] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:18 2022] PUT /identity/v3/projects/08a939739112446394c9bb8ff615fefd/users/b18ea04c50b047979b5d77babb60f84b/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:18.594010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.596346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 986/1947] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:18.663606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6114b05d-6b27-4651-bc31-012f59a32a57 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.669042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.669930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.670237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.670501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.671238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.672902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d0cdc7-3de8-47aa-83ff-8cb58a9b585f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.675337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:18.685698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 962/1948] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:18.689853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.704913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477d467-809c-4e3f-906b-5f17052270bd service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:18.709776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:18.710671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.710967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.711251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] PATH_INFO: `/v3/projects/08a939739112446394c9bb8ff615fefd/users/b18ea04c50b047979b5d77babb60f84b/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.713572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a939739112446394c9bb8ff615fefd, user_id=b18ea04c50b047979b5d77babb60f84b, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:18.716517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:18.736887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 987/1949] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:18 2022] PUT /identity/v3/projects/08a939739112446394c9bb8ff615fefd/users/b18ea04c50b047979b5d77babb60f84b/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:18.748771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:18.748771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:18.749026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:18.787702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] MFA Rules not processed for user `b18ea04c50b047979b5d77babb60f84b`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:18.789576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.907257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 963/1950] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:18.953314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.964953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:18.980025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:18.982178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.057750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4636f-bfd5-45b0-b719-4dd53624f5b5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.063503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.063794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a7f0a-164b-4499-8707-a95996da0b24 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.064416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.064706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.065140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] PATH_INFO: `/v3/projects/8ec0b997f16e47938eacaabe46031951` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.066805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec0b997f16e47938eacaabe46031951)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.068849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.069380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.070217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.070824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.071463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.085165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.088069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:19.088499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.107615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 964/1951] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:19.165308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec0b997f16e47938eacaabe46031951', 'request_id': 'req-f8c811ed-0e14-4d45-aa38-ae728232d81b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.177513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec0b997f16e47938eacaabe46031951', 'request_id': 'req-f8c811ed-0e14-4d45-aa38-ae728232d81b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.189351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec0b997f16e47938eacaabe46031951', 'request_id': 'req-f8c811ed-0e14-4d45-aa38-ae728232d81b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.212957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] The token cache is being invalidate because project 8ec0b997f16e47938eacaabe46031951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:19.213262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.214271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.220739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.236050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 988/1952] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:18 2022] DELETE /identity/v3/projects/8ec0b997f16e47938eacaabe46031951 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:19.237085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2a678-c60d-40f8-8d41-7b728bd1d899 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.293962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.295503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.330967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.331299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.331664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.332126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] PATH_INFO: `/v3/users/7952718961314833b9e0fd4e0523ab5e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.333395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7952718961314833b9e0fd4e0523ab5e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.335382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.373556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c811ed-0e14-4d45-aa38-ae728232d81b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.377459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:19.380457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.381389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.381706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.382027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] PATH_INFO: `/v3/users/28bede01326c4e5993162ddbbd0790fc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.383446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=28bede01326c4e5993162ddbbd0790fc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.386012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.398639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7952718961314833b9e0fd4e0523ab5e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.399913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7952718961314833b9e0fd4e0523ab5e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.412787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7952718961314833b9e0fd4e0523ab5e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.413692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7952718961314833b9e0fd4e0523ab5e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.420892 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7952718961314833b9e0fd4e0523ab5e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.437950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 965/1953] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:19 2022] DELETE /identity/v3/users/7952718961314833b9e0fd4e0523ab5e => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:19.437950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.437950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.441074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28bede01326c4e5993162ddbbd0790fc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.449634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28bede01326c4e5993162ddbbd0790fc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.466845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28bede01326c4e5993162ddbbd0790fc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.468125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28bede01326c4e5993162ddbbd0790fc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.469181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-72d205d2-8aba-4862-824a-e837c5dc7a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28bede01326c4e5993162ddbbd0790fc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.493339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 989/1954] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:19 2022] DELETE /identity/v3/users/28bede01326c4e5993162ddbbd0790fc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:19.501338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.501338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.501338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.534875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:19.536977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.541630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe134-2afd-4107-bff7-9eaec6401975 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.559837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:19.629778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 990/1955] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:19.643286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.645980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.649513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.650935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.651235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.651524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] PATH_INFO: `/v3/users/043616485a8543caa442320b69cd23d3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.652879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=043616485a8543caa442320b69cd23d3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.654991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.722198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '043616485a8543caa442320b69cd23d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.724138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '043616485a8543caa442320b69cd23d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.727720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb54f-e305-4699-9c57-7f5434d7ff3c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.732356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:19.733409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.734151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.735409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.735990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.738659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.744604 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '043616485a8543caa442320b69cd23d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.745968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '043616485a8543caa442320b69cd23d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.756469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '043616485a8543caa442320b69cd23d3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:19.757991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.760842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9054c3-9e36-4caf-86c0-6aca825c4bc0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:19.772448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 991/1956] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:56:19.785566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 966/1957] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:19 2022] DELETE /identity/v3/users/043616485a8543caa442320b69cd23d3 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:19.790561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.791300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.791707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.800437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.802522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.813350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:19.815217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.908948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 992/1958] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:19.919204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:19.921189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:19.967668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1f4f0-d0ff-4d41-85a6-3c2aca03e57f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:19.979862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:19.981360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:19.981855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:19.982359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] PATH_INFO: `/v3/projects/13f40c41ac634858a22f0106fa23e59b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:19.984455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f40c41ac634858a22f0106fa23e59b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:19.987406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.008688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6ee5d-c725-47c4-9338-53717abc658f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.018333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.019343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.019674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.019963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.038264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.041601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6325011-1ab7-4594-af96-324024ff1994 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.049018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f40c41ac634858a22f0106fa23e59b', 'request_id': 'req-73e63147-1d0a-43f2-90c8-203842dbde88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.055575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f40c41ac634858a22f0106fa23e59b', 'request_id': 'req-73e63147-1d0a-43f2-90c8-203842dbde88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.064081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f40c41ac634858a22f0106fa23e59b', 'request_id': 'req-73e63147-1d0a-43f2-90c8-203842dbde88'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.067549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 993/1959] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:20.083634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] The token cache is being invalidate because project 13f40c41ac634858a22f0106fa23e59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:20.084411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.088043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73e63147-1d0a-43f2-90c8-203842dbde88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.091328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 967/1960] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:19 2022] DELETE /identity/v3/projects/13f40c41ac634858a22f0106fa23e59b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:20.096477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.097082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.097082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.123509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:20.124985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.131434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:20.205317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 994/1961] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:20.213138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.213647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.214076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.235982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:20.237297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.251043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 968/1962] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:20.252917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.267927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a651d-a306-40b6-9f18-f4a0215ba609 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.276117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.277314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.277801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.284242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.285262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:20.287577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.288557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.310798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 995/1963] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:20.315318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.326237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0082a0b2-3a77-4fb6-b629-023afad826ef None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.331199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.331998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.332383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.332757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.334105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.337611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.339081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.341113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.356411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee94f1-3f0d-4371-b80d-84684517c4fa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.363221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.364559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.364977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.365329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] PATH_INFO: `/v3/projects/a3bd6c91b0bd48df886b7ee6fec0833c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.367246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3bd6c91b0bd48df886b7ee6fec0833c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.370253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.381045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 969/1964] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:20 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:20.386124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.433052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:20.485418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3bd6c91b0bd48df886b7ee6fec0833c', 'request_id': 'req-d3a5ce05-0761-4b0f-925b-38ccc9871a15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.510610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1610f3a-1d25-4d6f-8c51-c67d04bf912c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.515238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3bd6c91b0bd48df886b7ee6fec0833c', 'request_id': 'req-d3a5ce05-0761-4b0f-925b-38ccc9871a15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.528655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3bd6c91b0bd48df886b7ee6fec0833c', 'request_id': 'req-d3a5ce05-0761-4b0f-925b-38ccc9871a15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.556471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] The token cache is being invalidate because project a3bd6c91b0bd48df886b7ee6fec0833c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:20.557115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.557849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.564593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 996/1965] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:20 2022] DELETE /identity/v3/projects/a3bd6c91b0bd48df886b7ee6fec0833c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:20.566843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.568764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.631401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.632110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.632400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.632677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.634315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.637054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.654961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5ce05-0761-4b0f-925b-38ccc9871a15 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.660732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 970/1966] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:20 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:20.660901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.661796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.662086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.662369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] PATH_INFO: `/v3/users/b18ea04c50b047979b5d77babb60f84b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.663697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18ea04c50b047979b5d77babb60f84b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.665461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.666466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.667259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.684869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda1050-3c2f-422d-ad55-e140b46c743f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.692838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.693577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.693899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.694155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.694910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.697617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.705572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:20.717308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 971/1967] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:20.718529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.720328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.742045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30bb5a8-71a4-47e3-99e7-938cb2ac4885 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.746921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18ea04c50b047979b5d77babb60f84b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.750448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.752066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.752415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.752715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] PATH_INFO: `/v3/projects/950c39810a9d4e2083391886dac44016` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.754439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] RBAC: Authorizing `identity:delete_project(project_id=950c39810a9d4e2083391886dac44016)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.755280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18ea04c50b047979b5d77babb60f84b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.756420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:20.770752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18ea04c50b047979b5d77babb60f84b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.772227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18ea04c50b047979b5d77babb60f84b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.773317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18ea04c50b047979b5d77babb60f84b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.788120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 997/1968] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:20 2022] DELETE /identity/v3/users/b18ea04c50b047979b5d77babb60f84b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:20.788819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.792907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.824438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '950c39810a9d4e2083391886dac44016', 'request_id': 'req-5bd87c33-d706-4a81-8cc9-16d156fb0987'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.833372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '950c39810a9d4e2083391886dac44016', 'request_id': 'req-5bd87c33-d706-4a81-8cc9-16d156fb0987'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.842790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '950c39810a9d4e2083391886dac44016', 'request_id': 'req-5bd87c33-d706-4a81-8cc9-16d156fb0987'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.885455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] The token cache is being invalidate because project 950c39810a9d4e2083391886dac44016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:20.888698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.890183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:20.894222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eec7c8-1f71-425e-aa84-3aba2db690ef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:20.895181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 972/1969] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:20 2022] DELETE /identity/v3/projects/950c39810a9d4e2083391886dac44016 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:20.900634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:20.904670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:20.990877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:20.992110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:20.992400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:20.993406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:20.994345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:20.996406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.007046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:21.024438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 998/1970] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:21.025966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:21.028354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.044340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2048ee5-39c2-4c71-a89b-858b0161a8f8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.044530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd87c33-d706-4a81-8cc9-16d156fb0987 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.049480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.050667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.050986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.051514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.051933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.052108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] PATH_INFO: `/v3/projects/fabf63a579e8411994c908ad8e730b01/users/418468d27d9044ab8f5097f4871a4475/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.052226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.052497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] PATH_INFO: `/v3/users/1ed9594a669647129e1972939afe9f6c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.053874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed9594a669647129e1972939afe9f6c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.056734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.074834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabf63a579e8411994c908ad8e730b01, user_id=418468d27d9044ab8f5097f4871a4475, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.078619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888bc5-6042-4848-b1ba-7c7a4bd92091 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.096759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 999/1971] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:21 2022] PUT /identity/v3/projects/fabf63a579e8411994c908ad8e730b01/users/418468d27d9044ab8f5097f4871a4475/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:21.102119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.102410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.102693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.121019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed9594a669647129e1972939afe9f6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.121967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed9594a669647129e1972939afe9f6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.124562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:21.126464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.142000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed9594a669647129e1972939afe9f6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.142889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed9594a669647129e1972939afe9f6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.145572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:21.152080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed9594a669647129e1972939afe9f6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.185751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 973/1972] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:20 2022] DELETE /identity/v3/users/1ed9594a669647129e1972939afe9f6c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:21.185751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:21.186191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.219966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1000/1973] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:21.230219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:21.232888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.310057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d5509-40d8-44ff-8af9-51a3baf81923 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.316194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.317279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.317669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.318005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] PATH_INFO: `/v3/projects/08a939739112446394c9bb8ff615fefd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.318742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c736e8a1-fdae-43e8-9686-04866e1fe1d0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.319191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a939739112446394c9bb8ff615fefd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.321857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.388549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a939739112446394c9bb8ff615fefd', 'request_id': 'req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.400593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.401869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.402167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.402578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.403298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a939739112446394c9bb8ff615fefd', 'request_id': 'req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.415050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.416682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.419549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a939739112446394c9bb8ff615fefd', 'request_id': 'req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.422309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:21.431282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1001/1974] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:21.433829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:21.437318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.451387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] The token cache is being invalidate because project 08a939739112446394c9bb8ff615fefd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:21.451925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.452765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e9da5cf-1bc2-4ea1-b84e-4b85435c3d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.455401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 974/1975] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:21 2022] DELETE /identity/v3/projects/08a939739112446394c9bb8ff615fefd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:21.456358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9ccc-701c-4459-83f8-815dcde556d5 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.463136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.463456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.464125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.523595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:21.528740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.596236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.599935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.600255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.600551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] PATH_INFO: `/v3/users/03e441c8a93a4ae2a94c5cb13a4743cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.608192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03e441c8a93a4ae2a94c5cb13a4743cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.613359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.702801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:21.741430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 975/1976] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:21.748266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:21.754891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.769879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03e441c8a93a4ae2a94c5cb13a4743cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.784140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03e441c8a93a4ae2a94c5cb13a4743cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.786503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe08d9-03a3-4ef6-90bb-1392fad85289 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:21.810576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:21.811625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.811948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.812223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] PATH_INFO: `/v3/projects/8b88bb3e32c24be3a62b1ff05a0182cb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.819444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b88bb3e32c24be3a62b1ff05a0182cb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:21.821869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:21.828103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03e441c8a93a4ae2a94c5cb13a4743cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.829028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03e441c8a93a4ae2a94c5cb13a4743cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.830211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbba8aa9-9486-4ee9-b631-3f7a6c8421e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03e441c8a93a4ae2a94c5cb13a4743cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.862438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1002/1977] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:21 2022] DELETE /identity/v3/users/03e441c8a93a4ae2a94c5cb13a4743cf => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:21.867480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.867968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.868269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:21.916004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:21.917627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:21.922922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b88bb3e32c24be3a62b1ff05a0182cb', 'request_id': 'req-76379d67-b4bd-4741-a936-e43bf8ec4215'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.932481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b88bb3e32c24be3a62b1ff05a0182cb', 'request_id': 'req-76379d67-b4bd-4741-a936-e43bf8ec4215'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.947719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b88bb3e32c24be3a62b1ff05a0182cb', 'request_id': 'req-76379d67-b4bd-4741-a936-e43bf8ec4215'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.977997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] The token cache is being invalidate because project 8b88bb3e32c24be3a62b1ff05a0182cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:21.979013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.982359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-76379d67-b4bd-4741-a936-e43bf8ec4215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:21.995112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 976/1978] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:21 2022] DELETE /identity/v3/projects/8b88bb3e32c24be3a62b1ff05a0182cb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:21.997951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:21.998280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:21.998566 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.028870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:22.030068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.059808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1003/1979] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:22.061850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.064225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.088575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4081-4b5f-4952-b277-34a7cc1b176b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.095024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.095938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.096232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.096513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] PATH_INFO: `/v3/users/4e3e2ba37c534ded8382a19a318180b6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.097738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3e2ba37c534ded8382a19a318180b6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.099680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.137982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 977/1980] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:22.139133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.161602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4e679-9b62-468f-96ed-1e1be5e05838 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.168441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.181023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.181371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.182531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.183598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:22.184685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.186035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.190572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3e2ba37c534ded8382a19a318180b6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.203016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3e2ba37c534ded8382a19a318180b6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.206492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 978/1981] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:22.207707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.215216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.232049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3e2ba37c534ded8382a19a318180b6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.233085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3e2ba37c534ded8382a19a318180b6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.234522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3e2ba37c534ded8382a19a318180b6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.235888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:22.272905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1004/1982] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:22 2022] DELETE /identity/v3/users/4e3e2ba37c534ded8382a19a318180b6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:22.274933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.289893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.325477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:22.335629 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03ce3a-5b38-4e93-a250-66a1a7a2537a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.400570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.401293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.401586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.402133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.404449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.419257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858fa88-311f-4b21-acdc-e422b21a288f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.424907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.425628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.425972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.426221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.426338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.427069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.428322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.429338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.443050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 979/1983] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:22.462997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1005/1984] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:22 2022] POST /identity/v3/projects => generated 407 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:22.467375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.483480 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e1434-3025-4664-aaa4-98a62009ba35 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.577899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.579111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.579472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.579846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.580706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.583169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.613094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 980/1985] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:22 2022] POST /identity/v3/users => generated 304 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:22.618199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.634942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b70e1-71cb-487f-972c-0a6576e35b39 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.641957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.644227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.644584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.644880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.645753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.647720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.667858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1006/1986] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:22.668476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.690202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025de87c-fbcf-40a8-9098-d931b452a075 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.709281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.710365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.710803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.712022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.714157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.716945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.744627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.749256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:22.762560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 981/1987] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:22.767375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:22.775176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b481638-d5db-472f-875a-756431267f14 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.784200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.787331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.788608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.788941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] PATH_INFO: `/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.791750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e8c45e07a604465b0db6b1ac4439d0e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.807539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.825729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c13b8-9487-4c54-ba35-e031ea176036 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:22.853832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:22.854766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.855056 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.855341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] PATH_INFO: `/v3/projects/e80d4dcbcf9a4382987a2df79003adac/users/3a73cbf843ad49aea1c8a17cc89c6bc5/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.918805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e8c45e07a604465b0db6b1ac4439d0e', 'request_id': 'req-66d026c7-c32a-4df1-91ef-2ffe94b398bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.922107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80d4dcbcf9a4382987a2df79003adac, user_id=3a73cbf843ad49aea1c8a17cc89c6bc5, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:22.925136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a2111-6111-41ba-84f3-ec26b520610f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:22.926551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e8c45e07a604465b0db6b1ac4439d0e', 'request_id': 'req-66d026c7-c32a-4df1-91ef-2ffe94b398bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.944551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e8c45e07a604465b0db6b1ac4439d0e', 'request_id': 'req-66d026c7-c32a-4df1-91ef-2ffe94b398bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.960639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 982/1988] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:22 2022] PUT /identity/v3/projects/e80d4dcbcf9a4382987a2df79003adac/users/3a73cbf843ad49aea1c8a17cc89c6bc5/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:22.960639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27211d61-f502-4472-a506-346993aef191 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.960639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27211d61-f502-4472-a506-346993aef191 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.960639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27211d61-f502-4472-a506-346993aef191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:22.975241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] The token cache is being invalidate because project 8e8c45e07a604465b0db6b1ac4439d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:22.975549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.983186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-66d026c7-c32a-4df1-91ef-2ffe94b398bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:22.989595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1007/1989] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:22 2022] DELETE /identity/v3/projects/8e8c45e07a604465b0db6b1ac4439d0e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:22.991038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89831cd7-528c-4b85-977e-834f0f62c953 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:22.991220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89831cd7-528c-4b85-977e-834f0f62c953 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:22.991612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89831cd7-528c-4b85-977e-834f0f62c953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.037637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-27211d61-f502-4472-a506-346993aef191 None None] MFA Rules not processed for user `418468d27d9044ab8f5097f4871a4475`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:23.038272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-89831cd7-528c-4b85-977e-834f0f62c953 None None] MFA Rules not processed for user `3a73cbf843ad49aea1c8a17cc89c6bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:23.040353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-89831cd7-528c-4b85-977e-834f0f62c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.052229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27211d61-f502-4472-a506-346993aef191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.064542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:23.118452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:23.139379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1008/1990] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:22 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:23.139379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.139379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.139379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.167403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:23.169708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.197072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 983/1991] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:23.198072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27211d61-f502-4472-a506-346993aef191 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.201182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27211d61-f502-4472-a506-346993aef191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.275095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1009/1992] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:23.284169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.286604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.289073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27211d61-f502-4472-a506-346993aef191 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.294241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.295660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.296062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.296352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.297066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.298869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482c05e-55d5-4113-8816-0837ca86ec41 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.299176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.308509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.309218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.309523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.309899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.311124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.314043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1963e2a0-c84b-4282-a235-408359061368 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.335686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 984/1993] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/projects => generated 407 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:23.335686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.339901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.347694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1010/1994] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:23.347694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.347694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.348028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.369389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:23.370722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.411176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a755503-1587-45ad-8c74-4a19979b2e1d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.452477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1011/1995] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:23.453545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.461782 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:23.465588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc2d87-0a55-4fb3-870b-965d2d4c25e0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.472666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.473700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.473998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.474448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.475308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.478163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.493500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.494731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.495172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.495617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] PATH_INFO: `/v3/projects/04c9a6ccc7e44dd08f4d944f45a61ae6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.497084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04c9a6ccc7e44dd08f4d944f45a61ae6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.499691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.505215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1012/1996] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/users => generated 310 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:23.505215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.520491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a1f9f-b3a9-47ac-9cdf-6f642d5dcbb6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.527155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.528231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.528527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.528816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.529528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.532451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.566245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1013/1997] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/users => generated 293 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:23.566245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.572562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04c9a6ccc7e44dd08f4d944f45a61ae6', 'request_id': 'req-b04eb20a-582f-47bb-82eb-884905505377'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.578314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e597d39-c2bd-46ea-b779-443d4c3c5e8d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.580677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04c9a6ccc7e44dd08f4d944f45a61ae6', 'request_id': 'req-b04eb20a-582f-47bb-82eb-884905505377'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.586606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04c9a6ccc7e44dd08f4d944f45a61ae6', 'request_id': 'req-b04eb20a-582f-47bb-82eb-884905505377'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.612267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] The token cache is being invalidate because project 04c9a6ccc7e44dd08f4d944f45a61ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:23.612267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.612267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.612725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 985/1998] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:23 2022] DELETE /identity/v3/projects/04c9a6ccc7e44dd08f4d944f45a61ae6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:23.613522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.614834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.639980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:23.663619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.665058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.665202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.665477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.666230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:23.666544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.668212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.681118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1014/1999] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:23.682206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.684988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.702425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d24f61d-be34-4ee0-b309-dce3e73a6735 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.707537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.708438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.708794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.709035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.709778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.712038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.716296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04eb20a-582f-47bb-82eb-884905505377 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.721991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.722707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.723064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.723645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.723844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1015/2000] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:23.724322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.724682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.726144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.726323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.738402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 986/2001] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:23.739391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.741247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.746285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714282eb-6980-4325-9ede-a497d3b29699 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.751137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.752053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.752341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.752617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] PATH_INFO: `/v3/users/a75c789958a04542be471f9298bbd3b0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.754747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75c789958a04542be471f9298bbd3b0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.756749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.759079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2e3cb-a9e3-4c31-8444-ddb3311427cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.765224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:23.765224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:23.765695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:23.765906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:23.766695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:23.768777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:23.808430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 987/2002] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:23 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:23.808430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.826312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75c789958a04542be471f9298bbd3b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.834767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75c789958a04542be471f9298bbd3b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.860935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75c789958a04542be471f9298bbd3b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.862086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75c789958a04542be471f9298bbd3b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.862837 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75c789958a04542be471f9298bbd3b0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:23.878040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:23.882106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1016/2003] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:23 2022] DELETE /identity/v3/users/a75c789958a04542be471f9298bbd3b0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:23.883281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:23.884922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:23.909966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8340e37a-2a15-4eaf-8022-9cd2c2c56e21 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:23.993176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d507c80f-3f75-4239-a041-54875ca7c0ec admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.026518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.026518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.026518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.026518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] PATH_INFO: `/v3/projects/96baac1f9e70472fa3932c0261dfb81a/users/466b63ac6adb43918e5df4b7333aa700/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.027055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=96baac1f9e70472fa3932c0261dfb81a, user_id=466b63ac6adb43918e5df4b7333aa700, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.027055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.049518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.055412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.057681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.059936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1017/2004] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:23 2022] PUT /identity/v3/projects/96baac1f9e70472fa3932c0261dfb81a/users/466b63ac6adb43918e5df4b7333aa700/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:24.060111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] PATH_INFO: `/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5/users/78aa4c40604b4ac6a8a5a0f76470c860/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.064978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.070194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.102483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3873ef7d9d4779acde89bbcc3275c5, user_id=78aa4c40604b4ac6a8a5a0f76470c860, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.105471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.124958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:24.159574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 988/2005] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:23 2022] PUT /identity/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5/users/78aa4c40604b4ac6a8a5a0f76470c860/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:24.167679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.251343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aecda-cc2a-406d-8d3d-0c7cb04fbc5a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.284881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e36-bc7e-4660-83c3-f73dc7453d64 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.308228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.308930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.309233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.309539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.310354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.314016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.339959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 989/2006] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:24.344365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.347168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.362172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f645a36-d511-4753-b09b-e221d552cd2c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.380500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.381283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.381578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.381861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.382615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.385088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.426578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 990/2007] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:24.427414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.450807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630ff49-eb38-45a4-845e-0877255717d9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.463456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.464635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.465050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.465439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] PATH_INFO: `/v3/projects/96baac1f9e70472fa3932c0261dfb81a/users/466b63ac6adb43918e5df4b7333aa700/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.472874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96baac1f9e70472fa3932c0261dfb81a, user_id=466b63ac6adb43918e5df4b7333aa700, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.482593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.497768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.498473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.500415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.500415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.500415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.503047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.529801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 991/2008] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:24 2022] PUT /identity/v3/projects/96baac1f9e70472fa3932c0261dfb81a/users/466b63ac6adb43918e5df4b7333aa700/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:24.542202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.552426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1018/2009] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:24 2022] POST /identity/v3/users => generated 288 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:24.560939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.563766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.602787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d00102-dcc9-4906-af84-eb446acca138 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.669023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:24.692100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.692980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.693104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f865e-26b9-45a0-af38-efd14f5b8e10 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.693271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.693536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] PATH_INFO: `/v3/projects/084682327eec40d7b2c3c60bcd510942` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.700745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.701453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.701812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.702105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] PATH_INFO: `/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5/users/78aa4c40604b4ac6a8a5a0f76470c860/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.704339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb3873ef7d9d4779acde89bbcc3275c5, user_id=78aa4c40604b4ac6a8a5a0f76470c860, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.707259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034207d-1bff-4fc5-9196-67a74b2a399d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.708983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=084682327eec40d7b2c3c60bcd510942)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.710984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.725107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 992/2010] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:24 2022] PUT /identity/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5/users/78aa4c40604b4ac6a8a5a0f76470c860/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:24.728941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc6d6-ddda-4eb4-a6f8-e26be97ee748 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.728941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc6d6-ddda-4eb4-a6f8-e26be97ee748 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.728941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4cc6d6-ddda-4eb4-a6f8-e26be97ee748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.795715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2b4cc6d6-ddda-4eb4-a6f8-e26be97ee748 None None] MFA Rules not processed for user `466b63ac6adb43918e5df4b7333aa700`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:24.802003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2b4cc6d6-ddda-4eb4-a6f8-e26be97ee748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.813898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '084682327eec40d7b2c3c60bcd510942', 'request_id': 'req-faaec72a-8fca-4135-8eb0-afa7e8357a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:24.822195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '084682327eec40d7b2c3c60bcd510942', 'request_id': 'req-faaec72a-8fca-4135-8eb0-afa7e8357a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:24.824459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:24.829850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '084682327eec40d7b2c3c60bcd510942', 'request_id': 'req-faaec72a-8fca-4135-8eb0-afa7e8357a9e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:24.845672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] The token cache is being invalidate because project 084682327eec40d7b2c3c60bcd510942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:24.845966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:24.846794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:24.848474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1019/2011] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:24 2022] DELETE /identity/v3/projects/084682327eec40d7b2c3c60bcd510942 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:24.849299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.850609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.893009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 993/2012] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:24.894193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.894494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.894780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.919517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:24.920793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.945943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaec72a-8fca-4135-8eb0-afa7e8357a9e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:24.952082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:24.952769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:24.953281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:24.954184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:24.955461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:24.958296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:24.971865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1020/2013] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:24.971865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:24.973288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:24.999691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 994/2014] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:24.999691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.005861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34312c-d7c7-40bf-9e6b-59dfb12cce53 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.010798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.011466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.011762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.012076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.012826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.014440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ffcdf5-fd73-4bea-9c41-2157eb0f51cd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.028902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 995/2015] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:25.028902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.028902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.028902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.049840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453360a3-a783-4199-9f39-a7d6bd63c357 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.051217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:25.052741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:25.053569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.054228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.054520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.055412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.062344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 996/2016] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:25.066308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.068332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.070833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.078549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53da750-34b3-419d-ae3a-84dadeaa5363 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.087728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1021/2017] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:24 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:25.089500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.102532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.103744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc872f5-b044-4ea8-8267-80e461134a74 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.103879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.104103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.104399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] PATH_INFO: `/v3/projects/066ea829709f49ceb8eb696c35b40d7b/users/4c3d8fc4ad2c4d14b224e7f8dc3e9fb9/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.109696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.110535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.110832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.111162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.113097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:25.113433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.115111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.125100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=066ea829709f49ceb8eb696c35b40d7b, user_id=4c3d8fc4ad2c4d14b224e7f8dc3e9fb9, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.128511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ab059f-f940-41f4-91a5-98b963e2c135 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.132669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1022/2018] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:25.135303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.155858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 997/2019] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:25 2022] PUT /identity/v3/projects/066ea829709f49ceb8eb696c35b40d7b/users/4c3d8fc4ad2c4d14b224e7f8dc3e9fb9/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:25.156200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d339d-a94d-4385-9d73-be57e0f076e9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.162526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.163018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.163651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.233421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] MFA Rules not processed for user `4c3d8fc4ad2c4d14b224e7f8dc3e9fb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:25.237485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:25.276457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:25.307174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.309290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.309844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.317326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.325342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.332181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.358657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:25.391338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1023/2020] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:25.396141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.401429 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 998/2021] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:25.402606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.405343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:25.474082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f649ff-62d8-4398-aeaf-b436aa2ad812 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.475132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b4566-694a-4769-997f-f894281625bc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.480345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.481054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.481355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.481641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.482492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.484508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.521509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1024/2022] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:25.523706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.536873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78474df4-11b3-4bd0-8b0a-36363e159291 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.540451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.541370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.541839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.541957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.542380 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.542606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.542897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.543176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.544034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.545130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:25.545824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.557651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1025/2023] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:25.561606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.573823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bf7ef-08df-4b1c-9695-6c2ffc66991e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.579012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.580145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.580500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.580737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.581487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.583526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.599265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1026/2024] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:25.604350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.620122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f6a5e-638d-4ee2-bf34-255c8ae9f3df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.626191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.627190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.627491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.627869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] PATH_INFO: `/v3/projects/18b7b2c494c8445aa44275c4fc1bdd1b/users/466ce7a57b3040b5b03dff975f8e9cf7/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.640531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.642506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acc85e-2eb8-4449-b131-ee46c49cb448 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.649468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18b7b2c494c8445aa44275c4fc1bdd1b, user_id=466ce7a57b3040b5b03dff975f8e9cf7, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.653886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:25.656259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 999/2025] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:25.697915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.698368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1027/2026] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:25 2022] PUT /identity/v3/projects/18b7b2c494c8445aa44275c4fc1bdd1b/users/466ce7a57b3040b5b03dff975f8e9cf7/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:25.701744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.702214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.722921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.762730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] MFA Rules not processed for user `466ce7a57b3040b5b03dff975f8e9cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:25.764066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:25.788007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:25.857254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc319778-8c19-42f0-bf71-c1ec26ed8403 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:25.861618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:25.862359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:25.862976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:25.862976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:25.901124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1000/2027] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:25.904822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:25.983783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:25.994088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:25.996140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25127f-2358-4b91-9609-921b34b48281 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.008977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1028/2028] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:26.028378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.029493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.029872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.052103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a374-697b-468c-8f23-cddd33d8900a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.060984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.061641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.061933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.062371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.063231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.065699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.097268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] MFA Rules not processed for user `78aa4c40604b4ac6a8a5a0f76470c860`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:26.098610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:26.122543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1001/2029] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:25 2022] POST /identity/v3/projects => generated 373 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:26.124463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.284479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1029/2030] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:26.312386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673afc0c-2571-461b-a9d0-ecf19b735c61 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.318259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.338805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.338805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.339086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.339884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.345524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.382921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1002/2031] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:26 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:26.384238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.400653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102bfc8-f6e1-4611-84f7-bffc242a3ce1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.414123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.416483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.417281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.417701 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.420861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.425819 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.471001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.498951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1030/2032] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:26.498951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.538436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1563fa58-6a9f-4e41-ad06-a65f6161d880 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.552813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:26.558125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.561644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.561864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.562024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] PATH_INFO: `/v3/projects/6cd73048d41c4f1aa69cddf377669811/users/2c2e7cd329f948beb393eaa8abfd38aa/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.601116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd73048d41c4f1aa69cddf377669811, user_id=2c2e7cd329f948beb393eaa8abfd38aa, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.604482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320769e2-6d3d-4560-89b9-7e34fe2608f3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.610194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.612451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.613795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.614154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.614447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.623221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:26.640878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1031/2033] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:26 2022] PUT /identity/v3/projects/6cd73048d41c4f1aa69cddf377669811/users/2c2e7cd329f948beb393eaa8abfd38aa/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:26.644957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.720496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.722964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.737304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13df2c9-0ca2-4e1b-973f-d92c11089370 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.739382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1003/2034] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:26.747073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.747774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.748206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.748505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.749471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.751022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee9ca4-d2e5-4ffa-8764-35040d704c13 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.764246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1032/2035] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:26.768540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:26.780787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69f2e0-ec1f-4357-93f9-75114eae5453 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:26.789759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:26.790829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.791183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.792183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] PATH_INFO: `/v3/projects/6cd73048d41c4f1aa69cddf377669811/users/2c2e7cd329f948beb393eaa8abfd38aa/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.794534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cd73048d41c4f1aa69cddf377669811, user_id=2c2e7cd329f948beb393eaa8abfd38aa, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:26.797761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:26.813981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1004/2036] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:26 2022] PUT /identity/v3/projects/6cd73048d41c4f1aa69cddf377669811/users/2c2e7cd329f948beb393eaa8abfd38aa/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:26.825134 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:26.825558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:26.825789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:26.862740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] MFA Rules not processed for user `2c2e7cd329f948beb393eaa8abfd38aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:26.864014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:26.916248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:26.984259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1033/2037] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:26.997933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:27.066179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71acc911-7dec-4dce-8eab-2d5c5b96ca7b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:27.070025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:27.071160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:27.071464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:27.071859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:27.147729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:27.150339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:27.160473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1005/2038] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:28.219024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:28.231802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b0b31-5e5f-4524-9510-b4eff0469a2d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:28.235344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:28.236014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:28.236303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:28.236583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:28.301225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:28.302787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cef7c7-06d7-42e3-8778-87c2d52a89d0 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:28.314661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1034/2039] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:37.155812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:38.309205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=46 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:47.280569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:47.343320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f66ab-7da8-4e77-ab98-bd46c6568683 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:47.347913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:47.348780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:47.349067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:47.349346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] PATH_INFO: `/v3/users/4c3d8fc4ad2c4d14b224e7f8dc3e9fb9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:47.351132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3d8fc4ad2c4d14b224e7f8dc3e9fb9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:47.353080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:47.399114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3d8fc4ad2c4d14b224e7f8dc3e9fb9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.399712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3d8fc4ad2c4d14b224e7f8dc3e9fb9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.412692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3d8fc4ad2c4d14b224e7f8dc3e9fb9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.414107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3d8fc4ad2c4d14b224e7f8dc3e9fb9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.422052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3d8fc4ad2c4d14b224e7f8dc3e9fb9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.436837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1006/2040] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:47 2022] DELETE /identity/v3/users/4c3d8fc4ad2c4d14b224e7f8dc3e9fb9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:47.441475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:47.441978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:47.442068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:47.460887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:47.464101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:47.525914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1035/2041] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:47.535648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:47.537128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:47.596031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619362c-91d9-411e-a447-90a6d8ab95b0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:47.600287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:47.601052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:47.601372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:47.601679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:47.613185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:47.614715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f7f22-fac1-434e-8972-6d113495ccbf service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:47.625327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1007/2042] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:47.685614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:47.687052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:47.699777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd4499-ed89-4a6f-a021-873a2f0c4c4e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:47.704911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:47.705777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:47.706075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:47.706445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] PATH_INFO: `/v3/users/466ce7a57b3040b5b03dff975f8e9cf7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:47.707613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] RBAC: Authorizing `identity:delete_user(user_id=466ce7a57b3040b5b03dff975f8e9cf7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:47.709632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:47.804542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466ce7a57b3040b5b03dff975f8e9cf7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.823118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466ce7a57b3040b5b03dff975f8e9cf7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.845604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466ce7a57b3040b5b03dff975f8e9cf7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.847183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466ce7a57b3040b5b03dff975f8e9cf7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.848056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466ce7a57b3040b5b03dff975f8e9cf7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:47.872584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1036/2043] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:47 2022] DELETE /identity/v3/users/466ce7a57b3040b5b03dff975f8e9cf7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:47.877374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:47.877811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:47.878474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:47.901525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:47.903847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:47.931126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:47.999859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1008/2044] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:48.008560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.010992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.012845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.014439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.031392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3268dc34-5bad-4647-9627-a3c6b843517a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.042309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.044329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.045682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.046142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] PATH_INFO: `/v3/projects/066ea829709f49ceb8eb696c35b40d7b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.051676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] RBAC: Authorizing `identity:delete_project(project_id=066ea829709f49ceb8eb696c35b40d7b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.058662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.110162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7d673-847d-46f8-9347-265c6228dcd9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.114897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.115565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.115998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.116415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.128006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.129886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.131962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '066ea829709f49ceb8eb696c35b40d7b', 'request_id': 'req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.138919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '066ea829709f49ceb8eb696c35b40d7b', 'request_id': 'req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.140597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1009/2045] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:48.145300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '066ea829709f49ceb8eb696c35b40d7b', 'request_id': 'req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.151200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:48.163751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.168909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.169418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] The token cache is being invalidate because project 066ea829709f49ceb8eb696c35b40d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:48.169873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.170732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8c77f4ef-00c7-4471-a805-bfbbb3cfd49f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.173579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1037/2046] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:48 2022] DELETE /identity/v3/projects/066ea829709f49ceb8eb696c35b40d7b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:48.179565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7591448-4c10-4ddb-8af9-dc705caca414 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.180063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7591448-4c10-4ddb-8af9-dc705caca414 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.180836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7591448-4c10-4ddb-8af9-dc705caca414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.193049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b4922-45e4-41a9-868f-020a1ace0a34 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.209325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f7591448-4c10-4ddb-8af9-dc705caca414 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:48.211524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7591448-4c10-4ddb-8af9-dc705caca414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.295522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.296440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.296801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.297194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] PATH_INFO: `/v3/users/3a73cbf843ad49aea1c8a17cc89c6bc5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.299338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a73cbf843ad49aea1c8a17cc89c6bc5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.302793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.318896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1038/2047] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:48.329907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.330202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.330538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.363244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:48.364691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.383314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a73cbf843ad49aea1c8a17cc89c6bc5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.384039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a73cbf843ad49aea1c8a17cc89c6bc5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.384802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1039/2048] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:48.388628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.398962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb84b3c-66a9-4f6b-b715-7cd2fcc913d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.404608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.405279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.405561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.405838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.406570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:48.406998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.408337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a73cbf843ad49aea1c8a17cc89c6bc5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.408536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.409198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a73cbf843ad49aea1c8a17cc89c6bc5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.412370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:48.420705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-852b2bc7-c50b-44bc-93bd-a4a133e1a923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a73cbf843ad49aea1c8a17cc89c6bc5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.427570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1040/2049] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:56:48.432890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.435278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.443723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1010/2050] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:48 2022] DELETE /identity/v3/users/3a73cbf843ad49aea1c8a17cc89c6bc5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:48.448821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.449172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.449467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.452930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79a754-30ea-494c-b169-9f049ebb07d2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.470411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:48.471496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.542079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.542753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.543043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.543327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.544065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.545985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.548890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1011/2051] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:48.550408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.551807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.566268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25972b00-2481-4d90-a353-69cb7daf3823 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.571101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.572010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.572304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.572588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] PATH_INFO: `/v3/projects/18b7b2c494c8445aa44275c4fc1bdd1b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.573897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18b7b2c494c8445aa44275c4fc1bdd1b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.575574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1041/2052] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/projects => generated 367 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:48.575840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.576666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.579496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.628742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18b7b2c494c8445aa44275c4fc1bdd1b', 'request_id': 'req-39029b24-38b9-4417-9b34-82b94ec8f451'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.635058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18b7b2c494c8445aa44275c4fc1bdd1b', 'request_id': 'req-39029b24-38b9-4417-9b34-82b94ec8f451'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.640465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fdca7-b1ce-4018-ae3b-f9ccc8a347bf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.641960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18b7b2c494c8445aa44275c4fc1bdd1b', 'request_id': 'req-39029b24-38b9-4417-9b34-82b94ec8f451'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.654198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:48.654927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] The token cache is being invalidate because project 18b7b2c494c8445aa44275c4fc1bdd1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:48.655218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.655889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.657462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1012/2053] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:48 2022] DELETE /identity/v3/projects/18b7b2c494c8445aa44275c4fc1bdd1b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:48.658225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.659501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.698643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.699224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.699557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.699870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.701728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.726872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39029b24-38b9-4417-9b34-82b94ec8f451 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.731602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.732357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.732652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.733248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.734071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.736084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.751454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:48.760160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1013/2054] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:48.760903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.763798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.774029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.775492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.778131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3b074-edae-4c6d-9ba8-8e4adb2cfdd6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.782855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.783745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.784072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.784353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] PATH_INFO: `/v3/users/2c2e7cd329f948beb393eaa8abfd38aa` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.785594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1042/2055] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:48.785703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2e7cd329f948beb393eaa8abfd38aa)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.786259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.787579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.798198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682c923-7c99-4a17-88aa-f8254899c77e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.803273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.804796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.805093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.805379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.806117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.808202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.824021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1043/2056] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:48.825672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.839131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed5d24-d497-429d-bbe6-c06a6644cd62 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.852138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.852909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.853201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.853481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.854384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.856093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.861625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2e7cd329f948beb393eaa8abfd38aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.863698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2e7cd329f948beb393eaa8abfd38aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.869109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1044/2057] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:48.873540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:48.876110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:48.887131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2e7cd329f948beb393eaa8abfd38aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.888732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2e7cd329f948beb393eaa8abfd38aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.900343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3801e20-7c25-4a2d-a787-48ef5b871706 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:48.901153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c28c673-05cd-4444-a4c7-ba28dc5d4c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2e7cd329f948beb393eaa8abfd38aa'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:48.906469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:48.907376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.907723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.908114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] PATH_INFO: `/v3/projects/8e3df757b60e41009a03e11542ddf43c/users/e39d54b056dc4c2ca42c33e85391adf4/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:48.926005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1014/2058] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:48 2022] DELETE /identity/v3/users/2c2e7cd329f948beb393eaa8abfd38aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:48.929794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3df757b60e41009a03e11542ddf43c, user_id=e39d54b056dc4c2ca42c33e85391adf4, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:48.932674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:48.951603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1045/2059] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:48 2022] PUT /identity/v3/projects/8e3df757b60e41009a03e11542ddf43c/users/e39d54b056dc4c2ca42c33e85391adf4/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:48.962038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:48.962607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:48.963444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.036530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] MFA Rules not processed for user `e39d54b056dc4c2ca42c33e85391adf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:49.038301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.176094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1015/2060] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:49.192984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.208503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.255595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.257232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.345962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1344014-db09-4c79-8f85-1cd1047ecc86 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.354323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.356116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.356489 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.356694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.393521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dfb12-0977-40bb-8567-07038c0e7583 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.398538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.401991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.403375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.403766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.404477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.404836 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] PATH_INFO: `/v3/projects/e80d4dcbcf9a4382987a2df79003adac` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.406428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80d4dcbcf9a4382987a2df79003adac)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.408966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.413079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:49.430230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1046/2061] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:49.431287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.433095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.456233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057540b5-ee54-4170-84c1-bcbe6070c01c service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.466647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.467536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.467851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.471479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] PATH_INFO: `/v3/projects/6cd73048d41c4f1aa69cddf377669811` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.472537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cd73048d41c4f1aa69cddf377669811)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.475354 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.513811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:49.550640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80d4dcbcf9a4382987a2df79003adac', 'request_id': 'req-73f59c50-ac05-48ab-8a5e-4fc827949649'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.558195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80d4dcbcf9a4382987a2df79003adac', 'request_id': 'req-73f59c50-ac05-48ab-8a5e-4fc827949649'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.567075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80d4dcbcf9a4382987a2df79003adac', 'request_id': 'req-73f59c50-ac05-48ab-8a5e-4fc827949649'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.569481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cd73048d41c4f1aa69cddf377669811', 'request_id': 'req-b9bd6a89-5a7c-46b1-8d71-9a6766750179'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.578947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cd73048d41c4f1aa69cddf377669811', 'request_id': 'req-b9bd6a89-5a7c-46b1-8d71-9a6766750179'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.588961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cd73048d41c4f1aa69cddf377669811', 'request_id': 'req-b9bd6a89-5a7c-46b1-8d71-9a6766750179'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.608404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] The token cache is being invalidate because project 6cd73048d41c4f1aa69cddf377669811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:49.608672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] The token cache is being invalidate because project e80d4dcbcf9a4382987a2df79003adac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:49.608783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.609464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.609624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b9bd6a89-5a7c-46b1-8d71-9a6766750179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.610790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.612336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1047/2062] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:49 2022] DELETE /identity/v3/projects/6cd73048d41c4f1aa69cddf377669811 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:49.617858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6639a-d6a9-499c-a5ae-496de5634a70 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.618372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6639a-d6a9-499c-a5ae-496de5634a70 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.619015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6639a-d6a9-499c-a5ae-496de5634a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.623364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1016/2063] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:49 2022] DELETE /identity/v3/projects/e80d4dcbcf9a4382987a2df79003adac => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:49.624383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.632625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.649573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a1e6639a-d6a9-499c-a5ae-496de5634a70 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:49.651655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a1e6639a-d6a9-499c-a5ae-496de5634a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.727675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f59c50-ac05-48ab-8a5e-4fc827949649 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.754473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1048/2064] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:49.757093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.757582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.757705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.780238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:49.781464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.792683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1049/2065] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:49.796299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.804094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.806038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.806656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.807147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] PATH_INFO: `/v3/users/466b63ac6adb43918e5df4b7333aa700` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.807274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fae74-4d1b-441c-aefe-132e5c75e57d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.808744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=466b63ac6adb43918e5df4b7333aa700)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.811179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.813187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.814030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.814897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.815169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.816231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:49.816594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.818564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.839683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1050/2066] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:56:49.840928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.855327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c50b12-327c-41d3-b734-68855b532aa1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:49.864897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:49.865644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:49.866046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:49.866353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:49.867092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:49.869106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:49.884973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:49.900156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466b63ac6adb43918e5df4b7333aa700'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.901049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466b63ac6adb43918e5df4b7333aa700'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.915752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1051/2067] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:49 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:49.920267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:49.923075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:49.923647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466b63ac6adb43918e5df4b7333aa700'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.924647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466b63ac6adb43918e5df4b7333aa700'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.934539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466b63ac6adb43918e5df4b7333aa700'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:49.955234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1017/2068] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:49 2022] DELETE /identity/v3/users/466b63ac6adb43918e5df4b7333aa700 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:50.020330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8ef1-9198-4fa0-91d5-54c7e95e402a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.036827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:50.095964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.096658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.096954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.097241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.097954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.099937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.135660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.142071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1052/2069] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:49 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:50.147675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256c287-2b8c-4e1e-ba62-c839758b6bbb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.152536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.153216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.153511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.153798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.154547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.157116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.170201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1018/2070] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:50.174325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.186206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9f91c-4d2b-4991-88f7-76e37d0e3c7f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.192027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.192288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.193458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.193763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.194774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.196512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb8e1f-8cdc-4124-a0d5-2c6954ec47bc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.207641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1053/2071] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:50.212268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.224640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bc6b4-9f45-4720-8127-0b53487ef9e0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.230989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.231981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.232413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.232926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] PATH_INFO: `/v3/projects/41599a007e3e4c6d9f17fad3eb8abf05/users/1074e3aeff0c499b82d7be9018280584/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.254070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41599a007e3e4c6d9f17fad3eb8abf05, user_id=1074e3aeff0c499b82d7be9018280584, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.260165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.280183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1019/2072] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:50 2022] PUT /identity/v3/projects/41599a007e3e4c6d9f17fad3eb8abf05/users/1074e3aeff0c499b82d7be9018280584/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:50.292008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.292432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.292792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.332085 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] MFA Rules not processed for user `1074e3aeff0c499b82d7be9018280584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:50.336298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:50.512676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1054/2073] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:50.517798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.625120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c366c0-8169-436e-8d3c-cb3d94a5ea86 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.645261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.646026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.648709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.649148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.650288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.656155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.739266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1020/2074] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:50 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:50.747626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.790852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa520-c88a-41a2-9e44-7d7412a5222f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.796024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.798439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:50.870457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.871238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.871598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.871981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.872750 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.874888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.904117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1055/2075] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:50 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:50.905616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.917630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530002-7cac-496c-a23c-7053076db5e5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.917987 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf88b69-0450-4e99-b49c-089c041cc9c2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.923376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.923835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.924538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.924667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.924986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] PATH_INFO: `/v3/projects/96baac1f9e70472fa3932c0261dfb81a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.925129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.925683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.925860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.926279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96baac1f9e70472fa3932c0261dfb81a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.926969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.928965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.931451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:50.938291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:50.939222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:50.952028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1056/2076] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:50.954368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:50.977420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc466-ebb1-45db-93ca-60255ade9c0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:50.983794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:50.984746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:50.985822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:50.985822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:50.986363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:50.988194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.003650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1057/2077] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:51.009430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.021551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181a80c-821f-4bae-b360-04d25f4e39e5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.042989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96baac1f9e70472fa3932c0261dfb81a', 'request_id': 'req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:51.059517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96baac1f9e70472fa3932c0261dfb81a', 'request_id': 'req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:51.070467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96baac1f9e70472fa3932c0261dfb81a', 'request_id': 'req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:51.094092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] The token cache is being invalidate because project 96baac1f9e70472fa3932c0261dfb81a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:51.094525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:51.096257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e6cc8f40-9fa9-4ed2-9527-0cc6b9956c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:51.100855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1021/2078] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:50 2022] DELETE /identity/v3/projects/96baac1f9e70472fa3932c0261dfb81a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:51.105920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df93-0488-4dc3-a089-eb787807add4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.106236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df93-0488-4dc3-a089-eb787807add4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.106935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3df93-0488-4dc3-a089-eb787807add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.123969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.123969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.123969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.123969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] PATH_INFO: `/v3/projects/9266e89fed6c41f7bea04687e4a4db6e/users/1fe4d1fce2be4814a9c46034b43a9706/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.155511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-26c3df93-0488-4dc3-a089-eb787807add4 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:51.156818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-26c3df93-0488-4dc3-a089-eb787807add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.167676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9266e89fed6c41f7bea04687e4a4db6e, user_id=1fe4d1fce2be4814a9c46034b43a9706, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.170796 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbb27-ddfc-4a36-b039-3d56f93a0fbf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.175616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1022/2079] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:51.190811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.191421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.191808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.209372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1058/2080] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:51 2022] PUT /identity/v3/projects/9266e89fed6c41f7bea04687e4a4db6e/users/1fe4d1fce2be4814a9c46034b43a9706/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:51.221358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.223885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.223885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.233680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:51.234937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.259498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] MFA Rules not processed for user `1fe4d1fce2be4814a9c46034b43a9706`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:51.260771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.302807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1023/2081] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:51.307965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.318884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959abad-c2cd-48a2-addc-606708b74293 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.323907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.324681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.324960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.325239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.326007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:51.326315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.327933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.339496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1059/2082] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:51.341264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1024/2083] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:56:51.343789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.345130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.346846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.357775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115332-13c7-46f5-9485-f06736277f0e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.358912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863324bb-7db8-4f7b-8d09-e8ff0be1b724 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.362391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.363026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.363312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.363405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.363601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.364142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.364322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.364414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.364700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.365404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.367243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.367906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.396992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1025/2084] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:51.397201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1060/2085] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:56:51.398147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.398893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.441663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:51.459160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:51.466445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901cae4-5ab9-4c7e-9362-548cfe3da138 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.472904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.473518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.473808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.474099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.474799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.476717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a3886-4d26-47e2-bd76-86899abaf6e0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.476852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.481435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.482106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.482394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.482674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.483368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.485346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.505086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1026/2086] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:51.506172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.515908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1061/2087] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:51.517051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.517246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ccb65-f733-474a-a6b7-d5a561d85d10 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.521780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.523279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.523689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.524075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.524843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.526438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.527623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b84e23-081f-4873-b996-db755d59179b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.535473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.536302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.536594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.539912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.540213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1027/2088] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:51.540293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.540293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.540905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.550363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1062/2089] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:51.551030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d215c-bac0-4afd-8c0e-abbff003e83f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.554268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.556277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.556938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.557225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.557505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.558234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.560100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.565698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15306af6-e809-4734-9068-27b1c291bacc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.575287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.575505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1028/2090] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:51.575613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.575613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.575613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] PATH_INFO: `/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d/users/0d0fddfb2616494db0aad2045aa49304/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.587137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.592923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1c1f60f86042c5b41f5c6bfc385b9d, user_id=0d0fddfb2616494db0aad2045aa49304, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.596056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.602926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693bbe-0075-4274-abc2-9a9d36eb68d3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.611168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.612347 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.612781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.613218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] PATH_INFO: `/v3/projects/f5b0fb3b84354b8ab3e84e08ee090245/users/2b0924b467d64534bb57ed5ba84868d7/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.613870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1063/2091] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:51 2022] PUT /identity/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d/users/0d0fddfb2616494db0aad2045aa49304/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:56:51.617982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.632810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b0fb3b84354b8ab3e84e08ee090245, user_id=2b0924b467d64534bb57ed5ba84868d7, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.636733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bff43-8c97-4814-8f11-06862c7640ca admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.663193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1029/2092] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:51 2022] PUT /identity/v3/projects/f5b0fb3b84354b8ab3e84e08ee090245/users/2b0924b467d64534bb57ed5ba84868d7/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:51.676504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.676819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.677110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.708760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] MFA Rules not processed for user `2b0924b467d64534bb57ed5ba84868d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:51.710770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.714924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0d703-20e6-4496-a4a1-821ae86790e4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.800980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.802135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.802449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.803172 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.804811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.806464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.818648 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1064/2093] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:51.823766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.828753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1030/2094] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:51.842608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:51.844049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.846418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4ba0c-02e3-4493-9f76-b1ba145b3dcf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.855187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.856585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.856880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.857169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] PATH_INFO: `/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d/users/0d0fddfb2616494db0aad2045aa49304/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.859299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1c1f60f86042c5b41f5c6bfc385b9d, user_id=0d0fddfb2616494db0aad2045aa49304, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:51.862192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63830abd-bbb2-4998-9537-8d4d96e78c45 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:51.876413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1065/2095] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:51 2022] PUT /identity/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d/users/0d0fddfb2616494db0aad2045aa49304/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:51.887678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.888013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.888297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:51.908252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d5c51-a380-4d05-a78c-aaa9fe5d73be None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:51.918827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] MFA Rules not processed for user `0d0fddfb2616494db0aad2045aa49304`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:51.920265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:51.934477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:51.976801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:51.987106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:51.987788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:51.988114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:51.988394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.008147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1066/2096] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:52.024925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:52.026541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:52.040596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbd101-5604-4774-a339-eb0c98622cd3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:52.045824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:52.046716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:52.047224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:52.047724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.049772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:52.059133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:52.060624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:52.070009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1031/2097] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:56:52.089608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:52.119234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:52.120717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7e46e-628d-4460-87e9-562501e35bb1 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:52.132291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1067/2098] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:52.159162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746fc003-2d3a-4d75-b148-47cecd84606a service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:52.164261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:52.165128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:52.165420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:52.165699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] PATH_INFO: `/v3/users/2b0924b467d64534bb57ed5ba84868d7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.166915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0924b467d64534bb57ed5ba84868d7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:52.172553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:52.245215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0924b467d64534bb57ed5ba84868d7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:52.246238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0924b467d64534bb57ed5ba84868d7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:52.264066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0924b467d64534bb57ed5ba84868d7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:52.265145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0924b467d64534bb57ed5ba84868d7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:52.277167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0924b467d64534bb57ed5ba84868d7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:52.300340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1032/2099] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:52 2022] DELETE /identity/v3/users/2b0924b467d64534bb57ed5ba84868d7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:52.302485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:52.302485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:52.303568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.338078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:52.339603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:52.442716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1068/2100] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:52.450251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:52.451739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:52.486125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:52.560588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c7bbd-7629-4a67-a593-8d3a4bcf26d8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:52.565726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:52.566967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:52.567420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:52.567911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.588362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:52.589878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5a43e9-90d8-4606-9a59-23a629eac2ee service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:52.603660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1033/2101] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:52.910497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:52.912055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:52.933904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbd85c-fcb5-45d5-8afd-214c608ecd4b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:52.939008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:52.939948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:52.940328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:52.940515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] PATH_INFO: `/v3/projects/f5b0fb3b84354b8ab3e84e08ee090245` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:52.941791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b0fb3b84354b8ab3e84e08ee090245)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:52.944368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:52.997767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:53.002617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b0fb3b84354b8ab3e84e08ee090245', 'request_id': 'req-7382fad2-a091-4743-a989-5b9c4761e911'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:53.011297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b0fb3b84354b8ab3e84e08ee090245', 'request_id': 'req-7382fad2-a091-4743-a989-5b9c4761e911'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:53.017791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b0fb3b84354b8ab3e84e08ee090245', 'request_id': 'req-7382fad2-a091-4743-a989-5b9c4761e911'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:53.030876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] The token cache is being invalidate because project f5b0fb3b84354b8ab3e84e08ee090245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:53.031173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:53.031795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7382fad2-a091-4743-a989-5b9c4761e911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:53.033388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1069/2102] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:52 2022] DELETE /identity/v3/projects/f5b0fb3b84354b8ab3e84e08ee090245 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:53.039042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.039435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.039723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.058942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:53.060224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.119932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1034/2103] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:53.128909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.129195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.129477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.152351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:53.153566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.161899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1070/2104] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:53.166694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.176495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929546d-62b0-482d-8b2b-6244ea43354c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.181487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.182142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.182718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.183010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.184043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:53.184361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.186213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.203926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1035/2105] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:56:53.208241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.218595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace94d16-0ad7-4462-88c1-ebf94baea5cb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.225740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.226469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.226762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.227050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.227771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.229727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.257502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1071/2106] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:53 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:53.261597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.271271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d952a0-c484-4fa2-82ca-92fe93b952db admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.337781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.338454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.338743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.339021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.339748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.341745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.365412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1036/2107] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:53 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:53.372177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.411277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc29a-59a7-41a2-9ecd-914158c2ce9c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.424326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.425028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.425344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.425621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.426371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.428026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.445419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1072/2108] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:53.446935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.458904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc64d6-76cd-40f6-8579-c163fd819281 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.474107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.474792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.475088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.475366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.476146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.478218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c1077-300d-4514-928b-ab379a2de0da admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.502185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1037/2109] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:56:53.507077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.519097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334e70d-067f-4d4b-a8bf-dfb4c8c9f149 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.525462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.526364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.526660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.526946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] PATH_INFO: `/v3/projects/68628b41f15b4fefbc58343f27f01386/users/25d14bad835b4a9384705d4181177390/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.543597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68628b41f15b4fefbc58343f27f01386, user_id=25d14bad835b4a9384705d4181177390, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.546224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.562537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1073/2110] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:53 2022] PUT /identity/v3/projects/68628b41f15b4fefbc58343f27f01386/users/25d14bad835b4a9384705d4181177390/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:53.576610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.577168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.577168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.606964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] MFA Rules not processed for user `25d14bad835b4a9384705d4181177390`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:53.608210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.621445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:53.687261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1038/2111] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:53.699496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.701276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.773584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10258966-1213-4286-9f9b-14c4bfbc2422 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.777687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.778372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.778660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.778940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.789210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.790667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.800278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1074/2112] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:53.868050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:53.869501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.882445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0c0e-bc79-4ca5-a39e-a2ff8e2cc6ec None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:53.886135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:53.886800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:53.887088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:53.887369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:53.889267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:53.951414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:53.952893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:53.962521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1039/2113] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:54.590838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:54.599195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c412fe0-66b1-4240-b864-9600495a2a78 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:54.603434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:54.604113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:54.604403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:54.604681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:54.606452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:54.674310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:54.675779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:54.685718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1075/2114] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:54 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:56:55.809757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:55.811362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:55.896500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8511592-31ba-48dd-95bb-4f006c7b5577 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:55.901170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:55.902099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:55.902386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:55.902663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] PATH_INFO: `/v3/users/fae3b6e3b2e24719ad90614dc5c7e431` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:55.903870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae3b6e3b2e24719ad90614dc5c7e431)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:55.905789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:55.965412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae3b6e3b2e24719ad90614dc5c7e431'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:55.966125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae3b6e3b2e24719ad90614dc5c7e431'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:55.973137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:55.980209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae3b6e3b2e24719ad90614dc5c7e431'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:55.981054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae3b6e3b2e24719ad90614dc5c7e431'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:55.988184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-afe8f8d0-17c3-4be7-ba7a-5def5ca84c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae3b6e3b2e24719ad90614dc5c7e431'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.001703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1040/2115] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:56:55 2022] DELETE /identity/v3/users/fae3b6e3b2e24719ad90614dc5c7e431 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:56.179852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.186441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:56.192894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de782e-d6b8-4858-87e4-6ec746b57d9f service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.258776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.259644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.259963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.260246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] PATH_INFO: `/v3/projects/469bcf4beb3041559397e70538a7ea12` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.261416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=469bcf4beb3041559397e70538a7ea12)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.263320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.318306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '469bcf4beb3041559397e70538a7ea12', 'request_id': 'req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.325264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '469bcf4beb3041559397e70538a7ea12', 'request_id': 'req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.330845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '469bcf4beb3041559397e70538a7ea12', 'request_id': 'req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.343840 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] The token cache is being invalidate because project 469bcf4beb3041559397e70538a7ea12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:56:56.344147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.344794 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bf0d76b9-effa-4c44-8f7c-16114bf9bdb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:56:56.346511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1076/2116] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:56:56 2022] DELETE /identity/v3/projects/469bcf4beb3041559397e70538a7ea12 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:56.352697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.352991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.353270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.378582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:56.379894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:56.447615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1041/2117] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:56.455645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.455977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.456306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.474587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:56.476562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:56.485696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1077/2118] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:56:56.490204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.500691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e39ff-4ec4-49d1-89df-a6b14ddba8d4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.505665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.506365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.506659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.506946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.507690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:56:56.508031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.509639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.521876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1042/2119] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:56:56.526310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.535440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f90be-b2ec-4460-9e52-fd7cedbb41b9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.540672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.541328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.541643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.541956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.542665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.544839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.570847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1078/2120] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:56:56 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:56.574976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.584897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dac68-c343-4c1c-a669-fe917cada1fb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.642417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.643129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.643421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.643706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.644451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.646367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.666974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1043/2121] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:56:56 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:56:56.671118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.682158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dd7b22-c964-4db8-9e9b-30a7435d552f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.686660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.687314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.687599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.687907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.688639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.690266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.701878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1079/2122] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:56.705693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.716049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287101c-7aa1-493a-a670-59a5267ebacb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.720786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.721470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.721762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.722040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.722820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.724509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75514fe4-03a3-48ba-956d-d286c0901132 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.735739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1044/2123] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:56:56.739396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.750552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b08c9a3-5424-4983-ba7a-f236169086e3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.759561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.759561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.759561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.759561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] PATH_INFO: `/v3/projects/d3ebc40e12384ce4bb168690cfd5a27a/users/b3560616f4ec496f816602f9c73b69ef/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.774325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ebc40e12384ce4bb168690cfd5a27a, user_id=b3560616f4ec496f816602f9c73b69ef, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.777087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:56.798690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1080/2124] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:56:56 2022] PUT /identity/v3/projects/d3ebc40e12384ce4bb168690cfd5a27a/users/b3560616f4ec496f816602f9c73b69ef/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:56:56.804349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9ac8-754c-4a9a-9427-78813f2c41be None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.804642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9ac8-754c-4a9a-9427-78813f2c41be None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.804927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c9ac8-754c-4a9a-9427-78813f2c41be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.822313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-fc6c9ac8-754c-4a9a-9427-78813f2c41be None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:56:56.823513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fc6c9ac8-754c-4a9a-9427-78813f2c41be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:56.861519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:56.908058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1045/2125] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:56:56.918635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:56:56.920371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:56:56.954156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:56:56.980832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d97a3-ad42-4443-a64f-309574555de9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:56:56.985726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:56:56.986380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:56:56.986666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:56:56.986946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:56:56.996457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:56:56.997971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:56:57.009068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1081/2126] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:01.719178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077670d4-4244-4fa8-ae78-eb6487bbd9de None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:01.719640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077670d4-4244-4fa8-ae78-eb6487bbd9de None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:01.719879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077670d4-4244-4fa8-ae78-eb6487bbd9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:01.754090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-077670d4-4244-4fa8-ae78-eb6487bbd9de None None] MFA Rules not processed for user `b3560616f4ec496f816602f9c73b69ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:01.755340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-077670d4-4244-4fa8-ae78-eb6487bbd9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:01.829554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1046/2127] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:01.844616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:01.846259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:01.926665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ea83c-0da0-4b96-8d1d-16a58ec0c5ce service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:01.932833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:01.933470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:01.933761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:01.934043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:01.942479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:01.944143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:01.954012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1082/2128] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:01 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:03.434075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b442b1e-57a8-4ef7-8505-10986cb56b52 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:03.434999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b442b1e-57a8-4ef7-8505-10986cb56b52 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:03.435298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b442b1e-57a8-4ef7-8505-10986cb56b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:03.469907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3b442b1e-57a8-4ef7-8505-10986cb56b52 None None] MFA Rules not processed for user `b3560616f4ec496f816602f9c73b69ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:03.471146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3b442b1e-57a8-4ef7-8505-10986cb56b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:03.487284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1047/2129] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:10.944149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:10.945743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:10.965591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65400bc4-4090-4c3e-9342-12c35b3c7f5e service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:10.970513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:10.971408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:10.971749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:10.972205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] PATH_INFO: `/v3/users/1a4ab86ccb9a417aae0fbdae8a1bf239` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:10.973678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4ab86ccb9a417aae0fbdae8a1bf239)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:10.976464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:11.029654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4ab86ccb9a417aae0fbdae8a1bf239'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.037376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4ab86ccb9a417aae0fbdae8a1bf239'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.049828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4ab86ccb9a417aae0fbdae8a1bf239'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.050611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4ab86ccb9a417aae0fbdae8a1bf239'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.051155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4ab86ccb9a417aae0fbdae8a1bf239'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.066914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1083/2130] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:10 2022] DELETE /identity/v3/users/1a4ab86ccb9a417aae0fbdae8a1bf239 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:11.071668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:11.072209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:11.072292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:11.094851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:11.096229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:11.160199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1048/2131] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:11.171437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:11.172894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:11.207603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:11.234877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99efa5-7ec1-46dc-9825-99cddb6155b0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:11.238461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:11.239121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:11.239447 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:11.239740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:11.249104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:11.250549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:11.261118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1084/2132] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:11.572201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:11.573725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:11.588683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55195a9e-906c-4264-a0b1-41f6d003a5a9 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:11.594558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:11.595317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:11.595792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:11.596097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] PATH_INFO: `/v3/projects/ad95e0c83a9442b59b7bbde5ccf92674` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:11.597296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad95e0c83a9442b59b7bbde5ccf92674)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:11.599277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:11.647374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad95e0c83a9442b59b7bbde5ccf92674', 'request_id': 'req-27f3440f-74ad-47bf-966e-95209a595a13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.653569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad95e0c83a9442b59b7bbde5ccf92674', 'request_id': 'req-27f3440f-74ad-47bf-966e-95209a595a13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.657461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:11.659700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad95e0c83a9442b59b7bbde5ccf92674', 'request_id': 'req-27f3440f-74ad-47bf-966e-95209a595a13'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.673470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] The token cache is being invalidate because project ad95e0c83a9442b59b7bbde5ccf92674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:11.674098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.674746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.676358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1049/2133] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:11 2022] DELETE /identity/v3/projects/ad95e0c83a9442b59b7bbde5ccf92674 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:11.680355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:11.690233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effd8ccb-62c1-426b-8fa7-f22ac415718e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:11.757561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:11.758363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:11.758649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:11.758928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] PATH_INFO: `/v3/users/d88ef7c5b58a48038ebbf01692e867c2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:11.760141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88ef7c5b58a48038ebbf01692e867c2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:11.762067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:11.807346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88ef7c5b58a48038ebbf01692e867c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.814183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88ef7c5b58a48038ebbf01692e867c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.825942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88ef7c5b58a48038ebbf01692e867c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.826618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88ef7c5b58a48038ebbf01692e867c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.827237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86e28187-7156-43ec-a43a-ab20099593a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88ef7c5b58a48038ebbf01692e867c2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:11.840272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1085/2134] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:11 2022] DELETE /identity/v3/users/d88ef7c5b58a48038ebbf01692e867c2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:12.113382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.116664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.137680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3440f-74ad-47bf-966e-95209a595a13 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.214405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.215954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.216394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.216849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] PATH_INFO: `/v3/projects/2de42f5a1d404800b8464ed39db816c8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.218183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de42f5a1d404800b8464ed39db816c8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.220390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.276852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de42f5a1d404800b8464ed39db816c8', 'request_id': 'req-8e1a1345-7e31-4060-a06c-9db0b0056d65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.283319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de42f5a1d404800b8464ed39db816c8', 'request_id': 'req-8e1a1345-7e31-4060-a06c-9db0b0056d65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.290347 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de42f5a1d404800b8464ed39db816c8', 'request_id': 'req-8e1a1345-7e31-4060-a06c-9db0b0056d65'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.311622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] The token cache is being invalidate because project 2de42f5a1d404800b8464ed39db816c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:12.312018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.312737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-8e1a1345-7e31-4060-a06c-9db0b0056d65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.315140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1050/2135] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:12 2022] DELETE /identity/v3/projects/2de42f5a1d404800b8464ed39db816c8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:12.321904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.322397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.322627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.342388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:12.343635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.406314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:12.431056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1086/2136] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:12.432152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.432449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.432735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.453991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:12.455233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.469415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1051/2137] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:12.473367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.487510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51d1de-e9bc-445b-a9f6-43c1d5d34bcc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.498054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.498789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.499081 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.499480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.500177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:12.500487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.508623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.523860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1087/2138] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:12.528032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.539400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cf61-6ec6-471f-811f-b6b3c85f47c2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.545550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.546308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.546670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.547051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.547972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.549989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.579143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1052/2139] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:12.583824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.594755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5276736-7fef-4bd9-821f-e5b06a9b1fb5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.654686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.655361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.655655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.655977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.656677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.658600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.679583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1088/2140] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:12.683737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.694029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e48e0-93cb-4493-a174-58ca8a59b3cf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.698682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.699482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.699626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.699927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.700643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.702218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.714016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1053/2141] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:12.719864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.733849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ecb95-1dd2-4e5b-852f-48390994f0e8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.741557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.742540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.742828 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.743108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.743841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.745710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e16326-9293-44ae-b993-a3cca5014cef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.762350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1089/2142] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:12.766644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.778406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7087068-c48c-4dc5-b9dc-185a768a5b26 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.783337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.784259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.784578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.784833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] PATH_INFO: `/v3/projects/e987e2b1ceaf4a528d43f3336a2d1b59/users/8f0727fdd1e4439abf31cd249a9d29c1/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.802571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e987e2b1ceaf4a528d43f3336a2d1b59, user_id=8f0727fdd1e4439abf31cd249a9d29c1, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.805271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.818201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1054/2143] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:12 2022] PUT /identity/v3/projects/e987e2b1ceaf4a528d43f3336a2d1b59/users/8f0727fdd1e4439abf31cd249a9d29c1/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:12.830173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.830475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.830788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.842950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.844783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.859011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:12.860785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] MFA Rules not processed for user `8f0727fdd1e4439abf31cd249a9d29c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:12.862035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.914238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7684535-529c-44f5-906f-6d6570f791dc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.919205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.920138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.920395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.920676 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] PATH_INFO: `/v3/users/1074e3aeff0c499b82d7be9018280584` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.921882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1074e3aeff0c499b82d7be9018280584)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.923857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.936882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1090/2144] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:12.943321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.955334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981199fc-1cc5-424c-89ec-d8e4a41120df None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:12.960182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:12.960878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:12.961180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:12.961494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:12.962228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:12.964338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:12.981710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1074e3aeff0c499b82d7be9018280584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.982582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1074e3aeff0c499b82d7be9018280584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:12.993388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1091/2145] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:12.997202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:12.998847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:12.999768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1074e3aeff0c499b82d7be9018280584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.000495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1074e3aeff0c499b82d7be9018280584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.008374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-24a9daa5-63b9-45de-9aa4-e316e68f454a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1074e3aeff0c499b82d7be9018280584'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.022463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1055/2146] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:12 2022] DELETE /identity/v3/users/1074e3aeff0c499b82d7be9018280584 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:13.026869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.027105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.027393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.031560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:13.045693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:13.047272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.071573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed07cd-5258-4c56-8728-2fb450149743 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.109209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1056/2147] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:13.110494 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.113749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.125860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2560515-fdf5-44b8-a622-a06997d94908 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.131781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.132160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.132687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.132829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.132972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.133113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.133363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] PATH_INFO: `/v3/users/25d14bad835b4a9384705d4181177390` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.133466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.134076 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.134588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d14bad835b4a9384705d4181177390)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.136028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.136521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.155824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1092/2148] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:12 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:13.155824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.155824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.167198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:13.187434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d14bad835b4a9384705d4181177390'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.188513 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d14bad835b4a9384705d4181177390'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.201020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d14bad835b4a9384705d4181177390'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.201832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d14bad835b4a9384705d4181177390'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.209694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d14bad835b4a9384705d4181177390'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.218513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d384-3d8d-4261-8fe2-36e2aae5d29c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.222337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.222989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.223281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.223566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.224741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1057/2149] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:13 2022] DELETE /identity/v3/users/25d14bad835b4a9384705d4181177390 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:13.225602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.225708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.228627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.239086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.240825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91401593-945b-40cc-99ef-f59e8835d071 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.250164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1093/2150] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:13.251394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.251734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.252045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.273768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:13.275776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.299117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea853a0d-cf04-4677-9f4c-e4544f9f8e53 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.304411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.305099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.305423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.305703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.306717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.308621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.326446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1058/2151] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:13.327623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.344445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b13e0-bda3-4ccf-a805-32dd38881ce6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.351082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.352089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.352491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.352838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] PATH_INFO: `/v3/projects/6eda53f6d50547c593b11f8c51207881/users/0c5653360adc4166b59681019dd68223/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.365962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1094/2152] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:13.376663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.378970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.383471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eda53f6d50547c593b11f8c51207881, user_id=0c5653360adc4166b59681019dd68223, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.386992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.390505 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:13.409302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1059/2153] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:13 2022] PUT /identity/v3/projects/6eda53f6d50547c593b11f8c51207881/users/0c5653360adc4166b59681019dd68223/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:13.414232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.486836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297de1f-137a-41cf-839f-01b7706c1dd0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.526434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c8877-3659-417e-8826-8f9a944f918c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.532082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.532939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.533255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.533533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.534274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.535903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.555310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1060/2154] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:13.556850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.559721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.560427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.560724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.561015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.572622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030ff24-e48a-4163-bb95-5a064a566c98 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.574686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.576174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.577962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.578926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.579296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.579744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] PATH_INFO: `/v3/projects/6eda53f6d50547c593b11f8c51207881/users/0c5653360adc4166b59681019dd68223/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.581905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eda53f6d50547c593b11f8c51207881, user_id=0c5653360adc4166b59681019dd68223, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.585286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fe603-4c35-4ad4-bf53-a67cfaebacae admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.587691 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1095/2155] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:13.588509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.589960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.592173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:13.598111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1061/2156] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:13 2022] PUT /identity/v3/projects/6eda53f6d50547c593b11f8c51207881/users/0c5653360adc4166b59681019dd68223/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:13.606944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19f44f-4d13-4b26-9588-15e178adc39f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:13.610354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.610657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.610949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.670897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] MFA Rules not processed for user `0c5653360adc4166b59681019dd68223`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:13.672166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.693825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:13.694871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:13.695238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:13.695587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] PATH_INFO: `/v3/projects/41599a007e3e4c6d9f17fad3eb8abf05` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:13.696953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41599a007e3e4c6d9f17fad3eb8abf05)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:13.703605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:13.775746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1062/2157] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:13.798239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.801685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.811182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41599a007e3e4c6d9f17fad3eb8abf05', 'request_id': 'req-e0522d71-7e66-45aa-a583-35377276f5e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.829294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41599a007e3e4c6d9f17fad3eb8abf05', 'request_id': 'req-e0522d71-7e66-45aa-a583-35377276f5e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.846931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41599a007e3e4c6d9f17fad3eb8abf05', 'request_id': 'req-e0522d71-7e66-45aa-a583-35377276f5e1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.858495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:13.884713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] The token cache is being invalidate because project 41599a007e3e4c6d9f17fad3eb8abf05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:13.885058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.885803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:13.887459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1096/2158] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:13 2022] DELETE /identity/v3/projects/41599a007e3e4c6d9f17fad3eb8abf05 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:13.891283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:13.892788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:13.968952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91e932-f293-41d2-9805-cdebd73e38ca None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.009689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0522d71-7e66-45aa-a583-35377276f5e1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.023591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:14.024623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.024985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.025349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] PATH_INFO: `/v3/users/1fe4d1fce2be4814a9c46034b43a9706` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.027620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe4d1fce2be4814a9c46034b43a9706)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:14.030084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:14.083418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:14.084199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.085074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.085408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.089173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:14.133227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe4d1fce2be4814a9c46034b43a9706'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.155011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe4d1fce2be4814a9c46034b43a9706'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.183895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe4d1fce2be4814a9c46034b43a9706'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.184526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe4d1fce2be4814a9c46034b43a9706'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.186793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe4d1fce2be4814a9c46034b43a9706'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.214269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1097/2159] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:13 2022] DELETE /identity/v3/users/1fe4d1fce2be4814a9c46034b43a9706 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:14.219419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:14.221015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:14.222878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:14.249052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:14.252959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2366994-3c41-4997-b33d-46c229b8e861 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:14.264858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1063/2160] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:14.338723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7567a-98e4-4fab-b85c-c0e544a8f898 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.344513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:14.345536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.345827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.346122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] PATH_INFO: `/v3/projects/68628b41f15b4fefbc58343f27f01386` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.347560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=68628b41f15b4fefbc58343f27f01386)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:14.350324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:14.374867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:14.436932 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68628b41f15b4fefbc58343f27f01386', 'request_id': 'req-ce691fad-ab20-4dd7-ac81-b1555df4e19a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.453731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68628b41f15b4fefbc58343f27f01386', 'request_id': 'req-ce691fad-ab20-4dd7-ac81-b1555df4e19a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.465141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68628b41f15b4fefbc58343f27f01386', 'request_id': 'req-ce691fad-ab20-4dd7-ac81-b1555df4e19a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.508432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] The token cache is being invalidate because project 68628b41f15b4fefbc58343f27f01386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:14.509041 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.510078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce691fad-ab20-4dd7-ac81-b1555df4e19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:14.512825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1098/2161] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:14 2022] DELETE /identity/v3/projects/68628b41f15b4fefbc58343f27f01386 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:14.519858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.520342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.521408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.542240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:14.543864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:14.632001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1064/2162] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:14.642666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.642971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.643256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.686663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:14.689326 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:14.707576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1099/2163] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:14.709801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:14.726650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8850-1d0c-4cce-b0ef-a6f9bcd3f690 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.735477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:14.736325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.736618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.736946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.738154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:14.740190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:14.743530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:14.777858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1065/2164] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:14.779301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:14.797509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54383765-1be3-4bf5-a378-c71c04ee18a4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.813168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:14.813889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:14.814482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:14.814598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:14.819213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:14.822865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:14.892223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:14.892609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1100/2165] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:14 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:14.911542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f712f-0dd4-450e-9929-5eeb6c14f24e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:14.952408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:14.954198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.015081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.016123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.016516 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.016900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.017896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.020816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a32d66-e682-44f2-aea5-83545c5bb72e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.055795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1066/2166] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:14 2022] POST /identity/v3/users => generated 279 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:15.057441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.057838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.058223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.060483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1067/2167] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 13:57:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 13:57:15.061463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.063258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.081715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac06759-9d67-44f8-b510-54eb634a5036 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.084792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed732b8a-2ee5-43ce-a859-ac4db9aabc4f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.086787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.087655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.087981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.088274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] PATH_INFO: `/v3/projects/9266e89fed6c41f7bea04687e4a4db6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.089526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9266e89fed6c41f7bea04687e4a4db6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.090362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.091336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.091431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.091696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.092098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] PATH_INFO: `/v3/users/8f0727fdd1e4439abf31cd249a9d29c1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.093675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0727fdd1e4439abf31cd249a9d29c1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.096659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.150315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9266e89fed6c41f7bea04687e4a4db6e', 'request_id': 'req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.156074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0727fdd1e4439abf31cd249a9d29c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.156917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0727fdd1e4439abf31cd249a9d29c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.158312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9266e89fed6c41f7bea04687e4a4db6e', 'request_id': 'req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.165116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9266e89fed6c41f7bea04687e4a4db6e', 'request_id': 'req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.170841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:15.172482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0727fdd1e4439abf31cd249a9d29c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.173377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0727fdd1e4439abf31cd249a9d29c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.184005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] The token cache is being invalidate because project 9266e89fed6c41f7bea04687e4a4db6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:15.184347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.184729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0727fdd1e4439abf31cd249a9d29c1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.185220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.187571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1101/2168] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:14 2022] DELETE /identity/v3/projects/9266e89fed6c41f7bea04687e4a4db6e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:15.188555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.190669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.201535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:15.202251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1068/2169] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:15 2022] DELETE /identity/v3/users/8f0727fdd1e4439abf31cd249a9d29c1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:15.203251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.204674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.277369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d684db-bc5d-462b-a3cd-d1d1a5dcb047 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.282112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.282784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.283069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.283348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.284124 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.285730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc697d-feab-4884-ab50-5071fdd91c11 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.290803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214b167-d3b3-47f2-8f89-9d264fe6c747 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.297297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.300532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1102/2170] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:15.300532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f86b0-1d01-4704-bf56-7ee655914f31 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.300532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f86b0-1d01-4704-bf56-7ee655914f31 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.300532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f86b0-1d01-4704-bf56-7ee655914f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.302466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.303237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.303766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] PATH_INFO: `/v3/users/0d0fddfb2616494db0aad2045aa49304` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.305600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d0fddfb2616494db0aad2045aa49304)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.308104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.324397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-725f86b0-1d01-4704-bf56-7ee655914f31 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:15.325869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-725f86b0-1d01-4704-bf56-7ee655914f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.334884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1103/2171] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:15.336816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.337147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.337390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.363941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] MFA Rules not processed for user `4121712d68e243f1998602c91d1308d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:15.365443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.380530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d0fddfb2616494db0aad2045aa49304'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.381772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d0fddfb2616494db0aad2045aa49304'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.399103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d0fddfb2616494db0aad2045aa49304'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.400095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d0fddfb2616494db0aad2045aa49304'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.410354 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d0fddfb2616494db0aad2045aa49304'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:15.435440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1069/2172] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:15 2022] DELETE /identity/v3/users/0d0fddfb2616494db0aad2045aa49304 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:15.435440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.440831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.456423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1104/2173] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 13:57:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:15.457163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.459101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.530219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7be3c7-d983-4329-b29c-7294fdda1f63 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.544215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac87c0-ae0c-4126-8001-b368f5e9d97b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.550280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.550972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.551921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.551921 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.552767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.554447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.556013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:15.570440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1070/2174] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:15.571906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.574886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.593585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:15.602179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.603135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.603258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.605537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.608382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.625877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.628022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.648331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1105/2175] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:15.649353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.660747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bdd42-460b-40b8-8fa4-e4e8b013e7c0 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.667029 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.668106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.668492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.668867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] PATH_INFO: `/v3/projects/bbadc4bf96a7449a8e30e3d8eba7ad7c/users/4d7c6859bc034952b0c2f7ff31cb21da/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.693550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbadc4bf96a7449a8e30e3d8eba7ad7c, user_id=4d7c6859bc034952b0c2f7ff31cb21da, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.696824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc930d-1d50-45b4-9f41-6cdb4fd1a286 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.709556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d611739-7025-4b43-a99d-e7572b467bc3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.717492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1106/2176] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:15 2022] PUT /identity/v3/projects/bbadc4bf96a7449a8e30e3d8eba7ad7c/users/4d7c6859bc034952b0c2f7ff31cb21da/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:15.729305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.729706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.730584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.777379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] MFA Rules not processed for user `4d7c6859bc034952b0c2f7ff31cb21da`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:15.779806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.832115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.833521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.833963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.834104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.844040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.845543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.857035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1107/2177] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:15 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:15.857818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1071/2178] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:15.860971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.866203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.893913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:15.895343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:15.963866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6206816-5b2d-4459-99a9-71f9aebe6f54 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:15.968112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:15.968960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:15.969400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:15.969863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:15.980670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:15.982394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:15.994856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1072/2179] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:16.004273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f438d6e-c36f-47d4-9500-01bf829b446f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.011005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.011411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.012407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.012695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.012976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] PATH_INFO: `/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.014596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1c1f60f86042c5b41f5c6bfc385b9d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.017597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.022965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca36b4-a5fa-4a8f-b1de-f45fbfcb2446 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.027846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.028904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.029142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.029430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] PATH_INFO: `/v3/users/4d7c6859bc034952b0c2f7ff31cb21da` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.030639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7c6859bc034952b0c2f7ff31cb21da)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.032639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.069599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:16.090042 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1c1f60f86042c5b41f5c6bfc385b9d', 'request_id': 'req-37d86183-ce86-469c-925c-099ef7987e27'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.096352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1c1f60f86042c5b41f5c6bfc385b9d', 'request_id': 'req-37d86183-ce86-469c-925c-099ef7987e27'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.098598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7c6859bc034952b0c2f7ff31cb21da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.099365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7c6859bc034952b0c2f7ff31cb21da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.100370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:16.102816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1c1f60f86042c5b41f5c6bfc385b9d', 'request_id': 'req-37d86183-ce86-469c-925c-099ef7987e27'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.118694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7c6859bc034952b0c2f7ff31cb21da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.119726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7c6859bc034952b0c2f7ff31cb21da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.123785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] The token cache is being invalidate because project 4d1c1f60f86042c5b41f5c6bfc385b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:16.124251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.125111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-37d86183-ce86-469c-925c-099ef7987e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.127064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1108/2180] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:15 2022] DELETE /identity/v3/projects/4d1c1f60f86042c5b41f5c6bfc385b9d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:16.130538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e367d98f-4cc6-47e0-8730-0c65d8744d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7c6859bc034952b0c2f7ff31cb21da'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.133395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.134063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.134473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.149807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1073/2181] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:16 2022] DELETE /identity/v3/users/4d7c6859bc034952b0c2f7ff31cb21da => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:16.156320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fbc9f-efe7-452d-b0e5-2f925ce63470 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.156320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fbc9f-efe7-452d-b0e5-2f925ce63470 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.156320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fbc9f-efe7-452d-b0e5-2f925ce63470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.157667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:16.159094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.175240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8b5fbc9f-efe7-452d-b0e5-2f925ce63470 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:16.177296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8b5fbc9f-efe7-452d-b0e5-2f925ce63470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.256051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1109/2182] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:16.257146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.259297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.277938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee665232-68bd-4c62-bd71-6fd05815c6ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.282922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.286989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1074/2183] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:16.287139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.287139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.287139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] PATH_INFO: `/v3/users/418468d27d9044ab8f5097f4871a4475` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.287500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.287500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.287500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.288344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=418468d27d9044ab8f5097f4871a4475)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.290414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.305929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:16.308590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.322390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1075/2184] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:16.324730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.326367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.351855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418468d27d9044ab8f5097f4871a4475'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.361213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418468d27d9044ab8f5097f4871a4475'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.378890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418468d27d9044ab8f5097f4871a4475'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.380209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418468d27d9044ab8f5097f4871a4475'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.380925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418468d27d9044ab8f5097f4871a4475'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.396266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b33105-b380-4853-913a-25952c3bf96e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.396811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1110/2185] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:16 2022] DELETE /identity/v3/users/418468d27d9044ab8f5097f4871a4475 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:16.397635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.398949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.476400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58965cc8-dc7b-4c6b-a0d7-fc55e1f5d1e6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.489665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.490547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.491431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.491725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.492046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.497630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.497630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.497630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.497630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:16.497630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.498243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.499457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.501368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:16.519499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:16.522802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1111/2186] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:16.524012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.531614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.532180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.535350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.551182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1076/2187] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:16.552917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.561446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bea6c-2d12-41fe-9b3c-c192b3d2c75c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.564352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636bb3d-7e73-4967-a4c5-b8a0b6f8332d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.568012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.569621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.570059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.570495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] PATH_INFO: `/v3/projects/e987e2b1ceaf4a528d43f3336a2d1b59` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.572136 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e987e2b1ceaf4a528d43f3336a2d1b59)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.573278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.574217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.574612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.574773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.575079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.576319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.578917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.623387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1077/2188] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:16 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:16.628959 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.667864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e987e2b1ceaf4a528d43f3336a2d1b59', 'request_id': 'req-845af501-0e66-41ae-96cc-593ff864d0df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.681632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e987e2b1ceaf4a528d43f3336a2d1b59', 'request_id': 'req-845af501-0e66-41ae-96cc-593ff864d0df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.692910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e987e2b1ceaf4a528d43f3336a2d1b59', 'request_id': 'req-845af501-0e66-41ae-96cc-593ff864d0df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.725338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] The token cache is being invalidate because project e987e2b1ceaf4a528d43f3336a2d1b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:16.725855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.726996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:16.737275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1112/2189] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:16 2022] DELETE /identity/v3/projects/e987e2b1ceaf4a528d43f3336a2d1b59 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:16.738716 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.742599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf44ad-4df8-415b-b5a7-c5a4be0f4aa2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.743845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.847111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.848287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.849476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.850215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.851119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.854479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.895443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845af501-0e66-41ae-96cc-593ff864d0df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.901026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.902036 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.902557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.903009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] PATH_INFO: `/v3/users/0c5653360adc4166b59681019dd68223` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.904301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5653360adc4166b59681019dd68223)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.906445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:16.908234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1078/2190] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:16 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:16.909088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:16.910718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:16.923869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ac8bf-eb98-45a1-a539-9a89d20e2e77 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:16.929184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:16.929951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:16.930246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:16.930587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] PATH_INFO: `/v3/projects/fabf63a579e8411994c908ad8e730b01` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:16.931862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabf63a579e8411994c908ad8e730b01)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:16.933757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.002360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5653360adc4166b59681019dd68223'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.015947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5653360adc4166b59681019dd68223'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.051199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabf63a579e8411994c908ad8e730b01', 'request_id': 'req-18cb7796-726d-4965-bb3f-056977921f7d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.064388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5653360adc4166b59681019dd68223'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.065984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabf63a579e8411994c908ad8e730b01', 'request_id': 'req-18cb7796-726d-4965-bb3f-056977921f7d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.078564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5653360adc4166b59681019dd68223'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.081355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5653360adc4166b59681019dd68223'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.085350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabf63a579e8411994c908ad8e730b01', 'request_id': 'req-18cb7796-726d-4965-bb3f-056977921f7d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.088724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.094122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.107929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] The token cache is being invalidate because project fabf63a579e8411994c908ad8e730b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:17.109765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.113805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.117541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1079/2191] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:16 2022] DELETE /identity/v3/projects/fabf63a579e8411994c908ad8e730b01 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.121371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.122782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1113/2192] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:16 2022] DELETE /identity/v3/users/0c5653360adc4166b59681019dd68223 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.123474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.123842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.125164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.217811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd7b41-f619-4c5d-ba3d-5986cac5c9ad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.225184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb7796-726d-4965-bb3f-056977921f7d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.225409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.226645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.226954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.227644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] PATH_INFO: `/v3/projects/bbadc4bf96a7449a8e30e3d8eba7ad7c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.228660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbadc4bf96a7449a8e30e3d8eba7ad7c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.231467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.236264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.237615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.238118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.238737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.240525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.243680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.259437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1080/2193] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:17.261128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.263696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.302532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbadc4bf96a7449a8e30e3d8eba7ad7c', 'request_id': 'req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.303769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e19b-3705-48d4-9d62-2f0f876bed52 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.312371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbadc4bf96a7449a8e30e3d8eba7ad7c', 'request_id': 'req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.321088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbadc4bf96a7449a8e30e3d8eba7ad7c', 'request_id': 'req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.342564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] The token cache is being invalidate because project bbadc4bf96a7449a8e30e3d8eba7ad7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:17.343236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.344526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.351443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1114/2194] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:17 2022] DELETE /identity/v3/projects/bbadc4bf96a7449a8e30e3d8eba7ad7c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.352677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.354102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.391936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.393441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.393732 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.394016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] PATH_INFO: `/v3/users/78aa4c40604b4ac6a8a5a0f76470c860` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.395275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78aa4c40604b4ac6a8a5a0f76470c860)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.397468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.433995 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ce9ad-3e48-4029-8db3-7f912eabdaa8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.445292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.446289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.446403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.446674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.450009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.452071 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e33397-0fcb-42ea-80c3-3bc02248503c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.465312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1115/2195] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:17.466480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.466817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.467107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.478774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78aa4c40604b4ac6a8a5a0f76470c860'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.481049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78aa4c40604b4ac6a8a5a0f76470c860'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.504047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:17.509142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78aa4c40604b4ac6a8a5a0f76470c860'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.511838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.512176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78aa4c40604b4ac6a8a5a0f76470c860'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.520928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de1e9fe6-989a-4181-a2c4-f54b7db4c43f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78aa4c40604b4ac6a8a5a0f76470c860'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.521977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1116/2196] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:17.523426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.525467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.546028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a5287-5102-4f87-bee3-653f06bf77d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.551979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.552379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1081/2197] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:17 2022] DELETE /identity/v3/users/78aa4c40604b4ac6a8a5a0f76470c860 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.553871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1086e9-7a17-4675-9123-39b06b60d710 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.556811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1086e9-7a17-4675-9123-39b06b60d710 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.556811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1086e9-7a17-4675-9123-39b06b60d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.557107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.557107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.557107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] PATH_INFO: `/v3/projects/17405c4c4a834f4abe5db990a1ca1e90/users/d763b7fe664f4db6b2dd904b84cf7c0e/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.571850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.575799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=17405c4c4a834f4abe5db990a1ca1e90, user_id=d763b7fe664f4db6b2dd904b84cf7c0e, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.576483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-df1086e9-7a17-4675-9123-39b06b60d710 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:17.578156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df1086e9-7a17-4675-9123-39b06b60d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.579226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.588427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.604739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1117/2198] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:17 2022] PUT /identity/v3/projects/17405c4c4a834f4abe5db990a1ca1e90/users/d763b7fe664f4db6b2dd904b84cf7c0e/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.614434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.616499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.658216 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1082/2199] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:17.659632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.660033 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.660407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.693458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] MFA Rules not processed for user `d763b7fe664f4db6b2dd904b84cf7c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:17.695554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.701322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0654f-adc7-42e8-953a-755f64d1bcfc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.706513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.707400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.707671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.709590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] PATH_INFO: `/v3/projects/6eda53f6d50547c593b11f8c51207881` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.710878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eda53f6d50547c593b11f8c51207881)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.712848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.795220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eda53f6d50547c593b11f8c51207881', 'request_id': 'req-18416278-5df2-4127-acbc-bec78a206500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.799585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1083/2200] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:17.804149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.804700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eda53f6d50547c593b11f8c51207881', 'request_id': 'req-18416278-5df2-4127-acbc-bec78a206500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.811047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eda53f6d50547c593b11f8c51207881', 'request_id': 'req-18416278-5df2-4127-acbc-bec78a206500'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.815267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10efa9-94a5-4d29-aaf0-2b3c5443d0c1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.830553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] The token cache is being invalidate because project 6eda53f6d50547c593b11f8c51207881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:17.830896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.831483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.836708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1118/2201] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:17 2022] DELETE /identity/v3/projects/6eda53f6d50547c593b11f8c51207881 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:17.836708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.836708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.882064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.882866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.883292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.883693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.884545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:17.884922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.886590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.905522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1084/2202] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:17.905522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:17.908143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:17.911627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18416278-5df2-4127-acbc-bec78a206500 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.916848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:17.917793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:17.918163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:17.918525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] PATH_INFO: `/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:17.919826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb3873ef7d9d4779acde89bbcc3275c5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:17.921876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:17.948675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.949061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:17.973227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa59e95-6503-40d1-a8bd-e946bbdb2c77 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:17.976008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb3873ef7d9d4779acde89bbcc3275c5', 'request_id': 'req-f1240dfe-e011-4074-a290-d7e4901eb3e3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.983314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb3873ef7d9d4779acde89bbcc3275c5', 'request_id': 'req-f1240dfe-e011-4074-a290-d7e4901eb3e3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:17.989418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb3873ef7d9d4779acde89bbcc3275c5', 'request_id': 'req-f1240dfe-e011-4074-a290-d7e4901eb3e3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:18.003031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] The token cache is being invalidate because project fb3873ef7d9d4779acde89bbcc3275c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:18.003408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:18.004150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1240dfe-e011-4074-a290-d7e4901eb3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:18.005809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1119/2203] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:17 2022] DELETE /identity/v3/projects/fb3873ef7d9d4779acde89bbcc3275c5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:18.009997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.009997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.009997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.025889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.027097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.029441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.030090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.030379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.030644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.032410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.087308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1120/2204] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:18.088130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.089420 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.097061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.098596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3203d1c3-a2e5-48ca-93c8-ef0afe472475 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.101145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df3ca0-1e4a-41e9-9c9f-a6ca23e62378 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.105708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.106352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.106640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.106914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.107602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.107983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1085/2205] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:18.109248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.109544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.109733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa27a0-3222-4254-a75f-fa264a352573 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.109824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.131852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.133142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.142743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1121/2206] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.142743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccd15-bdc2-4f63-ab03-c9e85b517b2f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.142743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccd15-bdc2-4f63-ab03-c9e85b517b2f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.142743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccccd15-bdc2-4f63-ab03-c9e85b517b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.151796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1086/2207] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:18.151796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.163970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7e5fe-1e38-4ca4-8599-51d623500d95 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.175422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dccccd15-bdc2-4f63-ab03-c9e85b517b2f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.177487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dccccd15-bdc2-4f63-ab03-c9e85b517b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.238040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.238734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.239038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.239963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.240686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.242849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.259620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1122/2208] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:18.261270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.261659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.262040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.267302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1087/2209] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.268213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.279131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96491df-350f-4ce7-8d00-bcf613037126 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.284824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.285518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.286833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.287136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.287909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:18.288229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.288984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.289847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.290189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.299418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1123/2210] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:18.300370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.308612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1088/2211] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:18.309904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.315666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c8a8c-1195-4177-bb3a-016d702cba4a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.321666 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.322362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.322653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.322938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.323745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.325498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.326000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7b171d-8f80-4d11-b5c0-8a302db5c45a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.331031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.331707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.332207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.332371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.333251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:18.333592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.335357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.339157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1124/2212] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.340723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.345019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:18.349725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1089/2213] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:18.350868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.360685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0849ba-9d65-4097-bb5b-966f6e79fde0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.363179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada532a6-b2c9-4cd7-a1ca-8c732f7125bf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.369582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.370292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.370514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.370657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.370946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.371116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.371408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.371680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.371775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.372442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.373313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.374350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.387187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:18.387981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1090/2214] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.393025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.405515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55325c-38ea-4027-aa0d-93689b27b43f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.417249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1125/2215] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.417249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.417518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.417518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.417518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.417518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.417849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.417849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.448294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1091/2216] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.449815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.507357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c257f-0eea-4fe0-8755-2c2255e7ad0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.533151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a277b7e-6622-4f9a-8396-fb8e9218d1d3 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.539160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.539845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.540139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.540416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.541135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.543083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.567785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.568907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.569190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.569475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] PATH_INFO: `/v3/projects/73f8ae745d9d4d6b9391cc4cf54518db/users/b257de62315f4a03a6c0090a9156bfef/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.572418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1092/2217] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.573419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.584195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5cab3-3e7b-4ca5-b87d-916d81dfdd1d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.586424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f8ae745d9d4d6b9391cc4cf54518db, user_id=b257de62315f4a03a6c0090a9156bfef, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.589136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.589368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.589798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.590084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.590360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.591110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.593426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034545b-7cb3-46cc-b3b9-5f3f61b35ea3 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.603926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1126/2218] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:18 2022] PUT /identity/v3/projects/73f8ae745d9d4d6b9391cc4cf54518db/users/b257de62315f4a03a6c0090a9156bfef/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:18.611437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.625504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1093/2219] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/users => generated 286 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.627170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.627945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.628417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.665050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] MFA Rules not processed for user `b257de62315f4a03a6c0090a9156bfef`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.666493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.673996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c7c1-cb2c-4b7a-badb-434841cbfca3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.680455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.681215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.681566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.681947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.683165 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.684496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.698621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1127/2220] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.698621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.712766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00903c0b-8a7f-4f47-bd70-d0191887a3c4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.719551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.720281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.720568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.720886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.721571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.723173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.737077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1128/2221] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.737077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.745117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b04334-db86-43a7-9c80-c1d2ea1cb5f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.747370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1094/2222] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:18.748870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.749653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.750311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.750620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.750892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.751618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.753549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.755656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:18.760413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2fcec-7cfe-46a2-9b45-7052be3b7910 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.765472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.766136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.769645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.769786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1129/2223] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.769877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.769964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.770064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.770064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.779944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1095/2224] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:18.780123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5742c2-59c9-4d3d-b490-168784bc9ad8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.780609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.789728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.790401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.792966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.793267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.794292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.794491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec439-fcaf-43b9-a860-0e4316e332aa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.797508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.799267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.800182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.800475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.800940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] PATH_INFO: `/v3/projects/d2dd31498f7d4765946f557ef917dbdf/users/a0e7c03db2c24e81b75f95e0dede5d54/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.818395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2dd31498f7d4765946f557ef917dbdf, user_id=a0e7c03db2c24e81b75f95e0dede5d54, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.821188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.823473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:18.825579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1130/2225] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.826429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.839582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1096/2226] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:18 2022] PUT /identity/v3/projects/d2dd31498f7d4765946f557ef917dbdf/users/a0e7c03db2c24e81b75f95e0dede5d54/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:18.840432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:18.892709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d56bc1-d84a-42a0-b811-e6c89200c7fa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.904748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776dc6f-c05a-4e19-ad51-ffe30608a6a0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:18.909440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.910110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.910400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.910682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.912109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.914059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca1412-8dd4-4adc-a38f-b48e7d1466b1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.936217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1097/2227] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:18.937423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e4d2-a99a-4d4d-bc1a-f9e461342ad8 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.937720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e4d2-a99a-4d4d-bc1a-f9e461342ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.938001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28e4d2-a99a-4d4d-bc1a-f9e461342ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.959011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:18.960074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:18.960424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:18.960795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] PATH_INFO: `/v3/projects/d6c2b56305d240369d7b346ca7fbd26a/users/27cc29b393b44f20928eca7ac6c7bed9/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:18.975846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8b28e4d2-a99a-4d4d-bc1a-f9e461342ad8 None None] MFA Rules not processed for user `a0e7c03db2c24e81b75f95e0dede5d54`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:18.977067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8b28e4d2-a99a-4d4d-bc1a-f9e461342ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:18.977605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c2b56305d240369d7b346ca7fbd26a, user_id=27cc29b393b44f20928eca7ac6c7bed9, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:18.981273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:18.996441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1131/2228] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:18 2022] PUT /identity/v3/projects/d6c2b56305d240369d7b346ca7fbd26a/users/27cc29b393b44f20928eca7ac6c7bed9/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:18.997281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.055190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1098/2229] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:19.057223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.057524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.057807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.074390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3c4a62-5802-4900-8940-77fab0bf921c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.079875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.080757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.081194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.081741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.083279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.085245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.087017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] MFA Rules not processed for user `27cc29b393b44f20928eca7ac6c7bed9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:19.088269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:19.091390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:19.102182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1132/2230] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:19.103079 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.114303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c7d7c4-0769-4a9c-8afb-78f4da869589 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.121708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.122384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.122683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.122959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.123661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.125610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.158831 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1133/2231] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:19.159585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.164135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1099/2232] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:19.165173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.170275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:19.176209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698198fd-ab32-4de7-90af-497e029c5bca None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.236549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea470df1-9060-45d6-8454-701348554ddc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.242163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.243208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.243332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.243636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.243970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.244066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.244259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.244540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.244972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.245274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.246946 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.248038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.260732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1134/2233] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:19.262246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.264510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:19.274572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1100/2234] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:19.275322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.284985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530676-11f5-4b47-9469-edfa1c0d80aa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.289849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.290738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.291053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.291333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] PATH_INFO: `/v3/projects/df26de345652499ab776f7038707367d/users/eae936169ef44209a09f897159a515ed/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.306936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df26de345652499ab776f7038707367d, user_id=eae936169ef44209a09f897159a515ed, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.310238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.324912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1101/2235] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:19 2022] PUT /identity/v3/projects/df26de345652499ab776f7038707367d/users/eae936169ef44209a09f897159a515ed/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:19.325885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.335539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843fad-5afd-413a-8a64-d43f4e7735c0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.394997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc37dbd-97f6-4579-8c8f-174b0b60ed44 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.396738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.396738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.396738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.397176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.401280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.402335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.402605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.402993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.403748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.405443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e924003-48f6-4f29-87a4-45ce721d6b0b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.421433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1102/2236] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:19.423403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.423939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.424436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.450386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:19.454577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] MFA Rules not processed for user `eae936169ef44209a09f897159a515ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:19.456029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:19.465533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.467592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.476759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1135/2237] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:19.477645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.488221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf95c6a-7123-4c95-ab29-b7f34e221a7c service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.499785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.500504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.500800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.501089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.503859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.507807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.531559 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1136/2238] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:19.533640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.541880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:19.548980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8025ad-4d30-48b7-a36d-1bee590f049a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.555496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.557670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1103/2239] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:19.561058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.563850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.563850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] PATH_INFO: `/v3/projects/882bd853a88d4c9a9cab878311e08f78/users/7809203c3a0b4932a27c6c16aef1f67e/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.565623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.576703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a60383-fcc4-4a54-b055-b73e37fa2f1c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.578587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=882bd853a88d4c9a9cab878311e08f78, user_id=7809203c3a0b4932a27c6c16aef1f67e, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.581349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe194ba9-d120-463b-aaa9-4e7960190251 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.581456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.582169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.582525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.583506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.584821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.586798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.597612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1137/2240] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:19 2022] PUT /identity/v3/projects/882bd853a88d4c9a9cab878311e08f78/users/7809203c3a0b4932a27c6c16aef1f67e/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:19.609458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a649-a596-493f-b976-37ada0e65646 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.609763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a649-a596-493f-b976-37ada0e65646 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.610146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830a649-a596-493f-b976-37ada0e65646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.622257 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1104/2241] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:19.624071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.638324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9830a649-a596-493f-b976-37ada0e65646 None None] MFA Rules not processed for user `7809203c3a0b4932a27c6c16aef1f67e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:19.639547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9830a649-a596-493f-b976-37ada0e65646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:19.694298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365b1447-fea7-4d8a-8821-ca8b9b598ecb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.699223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.699940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.700238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.700539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.701589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.703554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.711928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1138/2242] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:19.716925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830a649-a596-493f-b976-37ada0e65646 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.729138 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1105/2243] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/users => generated 298 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:19.729880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830a649-a596-493f-b976-37ada0e65646 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.735344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.735496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.736163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.736458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.736741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.737452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.739628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.751062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3a6e2-f229-4057-9e57-94abdf1fca99 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.760802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.761560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.761831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.762165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.763098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.764763 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.776550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1106/2244] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:19.782916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1139/2245] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:19.783057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.783971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.796545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3cb88e-61d5-482d-a525-f457f4441e84 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.871889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271e6ee-f5f3-44c8-bca7-717b44c0df8e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.891720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.891720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.891720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.891720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.896338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.896338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.918491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.919514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.919810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.920140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] PATH_INFO: `/v3/projects/ccfdc19e632e4d11aac890d15b724538/users/e62583e51a2f4895857848e83b690dc9/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.931536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1140/2246] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:19 2022] POST /identity/v3/users => generated 296 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:19.936222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.950937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1381fc7-9f54-4de1-858a-27331b69e79c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.956546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:19.957233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:19.957527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:19.957850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:19.959260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.960818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:19.962496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:19.978736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1141/2247] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:19.983245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:19.988822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfdc19e632e4d11aac890d15b724538, user_id=e62583e51a2f4895857848e83b690dc9, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:19.995612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c753349-7789-4ec4-84e5-8b2ff3ab0fe6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:19.997122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.001338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.002908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.003485 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.004280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] PATH_INFO: `/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a/users/b08f7fdf224342549667fa823a615bd1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.030421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1107/2248] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:19 2022] PUT /identity/v3/projects/ccfdc19e632e4d11aac890d15b724538/users/e62583e51a2f4895857848e83b690dc9/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:20.033204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.045615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f358780aca4e6aa4f3a52c1b69a51a, user_id=b08f7fdf224342549667fa823a615bd1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.049514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.077283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1142/2249] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:19 2022] PUT /identity/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a/users/b08f7fdf224342549667fa823a615bd1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:20.081368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.104315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:20.154265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858d891-7544-4f86-8d5d-567976cc9859 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.183360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a275dda7-7844-4d52-b717-6d744b82b389 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.189647 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.190325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.190615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.190895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.191803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.193533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.213189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1143/2250] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:20.214646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.231387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e5f54a-e95d-425c-aa69-a41bac541270 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.236927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.238034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.238553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.239003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] PATH_INFO: `/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a/users/b08f7fdf224342549667fa823a615bd1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.242600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f358780aca4e6aa4f3a52c1b69a51a, user_id=b08f7fdf224342549667fa823a615bd1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.245947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57743d7-8b4e-4b14-bc12-09d084026b38 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.249258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.250280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.250740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.251187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.251977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.254186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.267863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1144/2251] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:20 2022] PUT /identity/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a/users/b08f7fdf224342549667fa823a615bd1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:20.277936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.278235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.278515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.287636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1108/2252] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:20.289430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.310375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c308bcc-baf7-4dc0-998b-0b63f87d44aa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.317921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] MFA Rules not processed for user `b08f7fdf224342549667fa823a615bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:20.319899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:20.407348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.408660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.409023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.409488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] PATH_INFO: `/v3/projects/ccfdc19e632e4d11aac890d15b724538/users/e62583e51a2f4895857848e83b690dc9/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.412041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfdc19e632e4d11aac890d15b724538, user_id=e62583e51a2f4895857848e83b690dc9, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.415475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42b2c0-1e09-4f40-bcfe-b679e17401d4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.431895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1109/2253] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:20 2022] PUT /identity/v3/projects/ccfdc19e632e4d11aac890d15b724538/users/e62583e51a2f4895857848e83b690dc9/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:20.444068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.444673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.445456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.471865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1145/2254] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:20.486299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.488169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:20.508359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] MFA Rules not processed for user `e62583e51a2f4895857848e83b690dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:20.508359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:20.554699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:20.572774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a3777a-7a26-4ba8-b6fb-d13e90554b7e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.613924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:20.634776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1110/2255] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:20.664519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.664911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.665152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.665434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.712729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.724006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e7f64-b68c-44ad-81ed-fb7e35d6b62c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.727752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.729221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.729221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.729221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.753917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.756387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.768320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1146/2256] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:20.792053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.793598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.807999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1111/2257] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:20.883919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:20.893091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9477f37-ce6d-4a86-8771-2ed6269bc62e service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:20.897363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:20.898262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:20.898554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:20.898832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:20.974467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:20.975990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:20.986427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1147/2258] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:23.296109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:23.299398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:23.391854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e6e480-a06e-4e2b-bdc4-69b728abfabf service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:23.402237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:23.403258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:23.403510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:23.404178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] PATH_INFO: `/v3/users/d763b7fe664f4db6b2dd904b84cf7c0e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:23.405116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d763b7fe664f4db6b2dd904b84cf7c0e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:23.407672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:23.430787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:23.439663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4301fe8-58dc-4016-ad12-b3387764ac77 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:23.443728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:23.444442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:23.444707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:23.444983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:23.464063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd763b7fe664f4db6b2dd904b84cf7c0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:23.464727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd763b7fe664f4db6b2dd904b84cf7c0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:23.480747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd763b7fe664f4db6b2dd904b84cf7c0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:23.481678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd763b7fe664f4db6b2dd904b84cf7c0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:23.490788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a9141a31-c0f3-4c9e-baa4-4dbbdb9f8f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd763b7fe664f4db6b2dd904b84cf7c0e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:23.492919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:23.521142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1112/2259] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:23 2022] DELETE /identity/v3/users/d763b7fe664f4db6b2dd904b84cf7c0e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:23.522802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:23.524314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:23.525712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:23.526087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:23.526446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:23.533660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1148/2260] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:23.544980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:23.546276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:23.558888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:23.626203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1113/2261] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:23.638866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:23.640366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:23.726009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6559d0-8107-4bf5-9473-a5e2fe533605 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:23.798425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:23.799331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:23.799835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:23.800168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:23.820624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:23.822284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:23.835938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1149/2262] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:24.165640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.168794 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.186562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb2f5f-b761-4539-9fba-fdaed689f339 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.193382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.194059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.194374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.194634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] PATH_INFO: `/v3/projects/17405c4c4a834f4abe5db990a1ca1e90` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.196118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17405c4c4a834f4abe5db990a1ca1e90)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.198048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.257471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17405c4c4a834f4abe5db990a1ca1e90', 'request_id': 'req-40d1b1ea-2ef2-419e-98b6-83532649fad9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:24.264888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17405c4c4a834f4abe5db990a1ca1e90', 'request_id': 'req-40d1b1ea-2ef2-419e-98b6-83532649fad9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:24.271567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.273055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.280294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17405c4c4a834f4abe5db990a1ca1e90', 'request_id': 'req-40d1b1ea-2ef2-419e-98b6-83532649fad9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:24.297572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] The token cache is being invalidate because project 17405c4c4a834f4abe5db990a1ca1e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:24.298218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:24.298810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-40d1b1ea-2ef2-419e-98b6-83532649fad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:24.300800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1114/2263] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:24 2022] DELETE /identity/v3/projects/17405c4c4a834f4abe5db990a1ca1e90 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:24.305683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d0055-bdce-4171-9c48-507b7b9d4c2a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.306383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d0055-bdce-4171-9c48-507b7b9d4c2a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.306678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d0055-bdce-4171-9c48-507b7b9d4c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.326786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-726d0055-bdce-4171-9c48-507b7b9d4c2a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:24.328059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-726d0055-bdce-4171-9c48-507b7b9d4c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.343213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8995eaf-0ac6-476a-a78c-0fc0bab29c96 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.393110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1115/2264] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:24.402303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.402601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.403054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.404149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.404818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.405110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.405393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.407195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.418918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:24.421945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:24.423353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.431432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1116/2265] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:24.435585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.445343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc0ddd-b89b-4fcd-9f35-01b1a8af5327 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.450117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.450791 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.451087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.451367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.452158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:24.452475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.454111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.468453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1117/2266] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:57:24.472626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.477188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.478650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.482844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977a90b-3459-4f81-a28a-d4908437423e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.487426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.488112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.488409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.488723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.489461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.490191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1150/2267] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:24.492401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.516155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:24.541923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1118/2268] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:24 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:24.546504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.560667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efbcf6-68b8-4f3d-8864-23dc58122160 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.632551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.633316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.633675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.634027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.634808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.637044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.663845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1151/2269] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:24 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:24.668399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.679901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6f22-27f7-4738-8dbb-079686e1cc3a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.684759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.685431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.685723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.686029 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.686777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.688415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.699414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1119/2270] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:24.703806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.717690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50634e-826e-4c6d-b017-0e5701a47c1e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.722632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.723625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.724045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.724236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.724977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.727031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c54c58-9199-4b6e-b212-9e746cb140b1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.741289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1152/2271] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:24.742479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.753520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088edd6-fc02-42b7-aa1f-e2557553fffe admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:24.758472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:24.759488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.759864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.760248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] PATH_INFO: `/v3/projects/456ea6bbe702420cb7fa4284088efda2/users/60bbd2f0609b4b8d81c9db106408eba8/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.784988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=456ea6bbe702420cb7fa4284088efda2, user_id=60bbd2f0609b4b8d81c9db106408eba8, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:24.788487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:24.803239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1120/2272] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:24 2022] PUT /identity/v3/projects/456ea6bbe702420cb7fa4284088efda2/users/60bbd2f0609b4b8d81c9db106408eba8/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:24.815580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8422-5472-40c5-b386-38d712d2c06a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:24.815580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8422-5472-40c5-b386-38d712d2c06a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:24.815580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8422-5472-40c5-b386-38d712d2c06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:24.846594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-31aa8422-5472-40c5-b386-38d712d2c06a None None] MFA Rules not processed for user `60bbd2f0609b4b8d81c9db106408eba8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:24.848011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31aa8422-5472-40c5-b386-38d712d2c06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:24.919965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1153/2273] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:24.964141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:24.966023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:25.028697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffaf5349-2585-4f3e-96ee-3574d3f75a0a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:25.032461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:25.033180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:25.033537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:25.033889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:25.049845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:25.051558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:25.060975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1121/2274] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:31.648154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac4dc-2f5a-4ecc-a954-1f4f83f2fc14 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:31.648702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac4dc-2f5a-4ecc-a954-1f4f83f2fc14 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:31.648702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ac4dc-2f5a-4ecc-a954-1f4f83f2fc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:31.670465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:31.676428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b1ac4dc-2f5a-4ecc-a954-1f4f83f2fc14 None None] MFA Rules not processed for user `60bbd2f0609b4b8d81c9db106408eba8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:31.678255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b1ac4dc-2f5a-4ecc-a954-1f4f83f2fc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:31.701905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1154/2275] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:33.685486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:33.687024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:33.826660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2477e7-5345-4d3a-8643-8c91cd17dd06 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:33.832450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:33.835413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:33.839477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:33.839790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] PATH_INFO: `/v3/users/a0e7c03db2c24e81b75f95e0dede5d54` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:33.845146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e7c03db2c24e81b75f95e0dede5d54)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:33.850315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:33.867497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:33.930103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e7c03db2c24e81b75f95e0dede5d54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:33.930103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e7c03db2c24e81b75f95e0dede5d54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:33.951534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e7c03db2c24e81b75f95e0dede5d54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:33.952484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e7c03db2c24e81b75f95e0dede5d54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:33.965656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e7c03db2c24e81b75f95e0dede5d54'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:33.983176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1122/2276] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:33 2022] DELETE /identity/v3/users/a0e7c03db2c24e81b75f95e0dede5d54 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:33.987977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:33.988896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:33.989423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:34.019621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:34.021896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:34.088965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1155/2277] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:34.098593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:34.100156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:34.257213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07644a4-78a0-4bd4-888a-4c6a53a1ac22 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:34.262489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:34.263080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:34.263375 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:34.263889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:34.283931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:34.287627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:34.299917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1123/2278] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:35.062991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:35.064514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:35.085405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577f109-a383-4e2c-8992-1a5716f0a800 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:35.098268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:35.101982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:35.102507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:35.103317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] PATH_INFO: `/v3/projects/d2dd31498f7d4765946f557ef917dbdf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:35.106466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2dd31498f7d4765946f557ef917dbdf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:35.108721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:35.167689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2dd31498f7d4765946f557ef917dbdf', 'request_id': 'req-ce09e28a-0f6a-4990-80ea-36924d822cd3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.174099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2dd31498f7d4765946f557ef917dbdf', 'request_id': 'req-ce09e28a-0f6a-4990-80ea-36924d822cd3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.180912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2dd31498f7d4765946f557ef917dbdf', 'request_id': 'req-ce09e28a-0f6a-4990-80ea-36924d822cd3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.197858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] The token cache is being invalidate because project d2dd31498f7d4765946f557ef917dbdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:35.198252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.198943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.200882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1156/2279] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:35 2022] DELETE /identity/v3/projects/d2dd31498f7d4765946f557ef917dbdf => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:35.204854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:35.225261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd2482-cf2c-4163-81e5-eb22b0d493df service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:35.289055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:35.297938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:35.299180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:35.299671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:35.299991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] PATH_INFO: `/v3/users/7809203c3a0b4932a27c6c16aef1f67e` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:35.302122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7809203c3a0b4932a27c6c16aef1f67e)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:35.305240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:35.388864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7809203c3a0b4932a27c6c16aef1f67e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.389535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7809203c3a0b4932a27c6c16aef1f67e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.407061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7809203c3a0b4932a27c6c16aef1f67e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.407808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7809203c3a0b4932a27c6c16aef1f67e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.415973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7809203c3a0b4932a27c6c16aef1f67e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.430518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1124/2280] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:35 2022] DELETE /identity/v3/users/7809203c3a0b4932a27c6c16aef1f67e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:35.762790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:35.764314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:35.784149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09e28a-0f6a-4990-80ea-36924d822cd3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:35.886665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:35.887752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:35.888657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:35.895709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] PATH_INFO: `/v3/projects/882bd853a88d4c9a9cab878311e08f78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:35.897480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=882bd853a88d4c9a9cab878311e08f78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:35.899642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:35.901549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:35.986412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '882bd853a88d4c9a9cab878311e08f78', 'request_id': 'req-a62c731c-972e-4612-af0d-1f828d6acb61'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:35.993816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '882bd853a88d4c9a9cab878311e08f78', 'request_id': 'req-a62c731c-972e-4612-af0d-1f828d6acb61'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.000651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '882bd853a88d4c9a9cab878311e08f78', 'request_id': 'req-a62c731c-972e-4612-af0d-1f828d6acb61'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.016016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] The token cache is being invalidate because project 882bd853a88d4c9a9cab878311e08f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:36.016381 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.017051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.018599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1157/2281] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:35 2022] DELETE /identity/v3/projects/882bd853a88d4c9a9cab878311e08f78 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:36.023674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:36.025330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:36.040294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19052d-b5b9-4a64-b709-36c07f93a9f7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:36.112644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:36.113596 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.113991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.114514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] PATH_INFO: `/v3/users/b08f7fdf224342549667fa823a615bd1` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.115842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08f7fdf224342549667fa823a615bd1)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:36.117839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:36.173832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08f7fdf224342549667fa823a615bd1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.174542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08f7fdf224342549667fa823a615bd1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.190577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08f7fdf224342549667fa823a615bd1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.191413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08f7fdf224342549667fa823a615bd1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.199134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-39529d7e-55d4-4582-a142-f3e7e896741e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08f7fdf224342549667fa823a615bd1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.202856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:36.215909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1125/2282] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:36 2022] DELETE /identity/v3/users/b08f7fdf224342549667fa823a615bd1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:36.499612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:36.501192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:36.523629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c731c-972e-4612-af0d-1f828d6acb61 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:36.581810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:36.582562 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.582851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.583136 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] PATH_INFO: `/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.584386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f358780aca4e6aa4f3a52c1b69a51a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:36.588771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:36.647952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f358780aca4e6aa4f3a52c1b69a51a', 'request_id': 'req-e8de7f18-6054-4532-9fd5-19fecde62620'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.654909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f358780aca4e6aa4f3a52c1b69a51a', 'request_id': 'req-e8de7f18-6054-4532-9fd5-19fecde62620'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.661468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f358780aca4e6aa4f3a52c1b69a51a', 'request_id': 'req-e8de7f18-6054-4532-9fd5-19fecde62620'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.677532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] The token cache is being invalidate because project d9f358780aca4e6aa4f3a52c1b69a51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:36.677901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.678566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e8de7f18-6054-4532-9fd5-19fecde62620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:36.684920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1158/2283] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:36 2022] DELETE /identity/v3/projects/d9f358780aca4e6aa4f3a52c1b69a51a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:36.686402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28408420-5715-4859-beba-f409f6dc5285 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.686896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28408420-5715-4859-beba-f409f6dc5285 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.687282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28408420-5715-4859-beba-f409f6dc5285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.731902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-28408420-5715-4859-beba-f409f6dc5285 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:36.733274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28408420-5715-4859-beba-f409f6dc5285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:36.807867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1126/2284] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:36.818477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.819131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.819131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.838769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:36.840175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:36.848523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1159/2285] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:36.853142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28408420-5715-4859-beba-f409f6dc5285 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:36.865003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28408420-5715-4859-beba-f409f6dc5285 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:36.871327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:36.872052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.872368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.872626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.873445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:36.873826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:36.875524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:36.894946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1127/2286] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:36.896582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:36.906332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354d071-3c0c-4a3e-a535-d167a9143a0e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:36.913078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:36.913856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:36.914232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:36.914582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:36.915366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:36.917479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:36.929629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:36.952414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1160/2287] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:36 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:36.954856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:36.965753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed2cf4-1107-416b-be6b-3da81f457596 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.065431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.068675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.069076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.069462 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.070326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.072540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.099849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1128/2288] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:36 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:37.100278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.110973 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114bcd-885d-44a1-b7be-55470e5ca6e5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.115873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.116665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.117058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.117426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.118262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.119992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.134625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1161/2289] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:37.138935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.152164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444aff5-0d2b-48c3-8ea5-c1abe63f70cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.158519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.159218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.159582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.160277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.160870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.162548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f6d1b-210f-4d83-9ab1-20a2a38f3827 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.173587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1129/2290] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:37.177711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.189900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d13662-79cd-4434-bdbf-485a617026ca admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.200300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.201334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.201633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.201925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] PATH_INFO: `/v3/projects/39e8c4e4dc8c4d28afb69497ad759e63/users/83344606769f4639a527880b8fb77f09/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.226795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e8c4e4dc8c4d28afb69497ad759e63, user_id=83344606769f4639a527880b8fb77f09, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.230255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.250613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1162/2291] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:37 2022] PUT /identity/v3/projects/39e8c4e4dc8c4d28afb69497ad759e63/users/83344606769f4639a527880b8fb77f09/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:37.265822 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.266272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.266549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.330889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.333346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:37.335426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] MFA Rules not processed for user `83344606769f4639a527880b8fb77f09`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:37.374702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:37.449587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:37.503745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1130/2292] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:37.506660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.514734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d34b2e-73aa-41bc-9fa9-fa3f6d992943 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.521169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3915c-1c17-4eb9-8e0a-70f7ad288702 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.533583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.534009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.534497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.534791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.535074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] PATH_INFO: `/v3/users/b257de62315f4a03a6c0090a9156bfef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.535617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.536817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.538441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.539367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.541857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.542202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b257de62315f4a03a6c0090a9156bfef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.544202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.611321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1131/2293] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:37 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:37.613015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.677329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b257de62315f4a03a6c0090a9156bfef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:37.685249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b257de62315f4a03a6c0090a9156bfef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:37.702771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b257de62315f4a03a6c0090a9156bfef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:37.703712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b257de62315f4a03a6c0090a9156bfef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:37.704387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8518eb64-61df-47d4-a730-3a264c3c10f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b257de62315f4a03a6c0090a9156bfef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:37.727749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1163/2294] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:37 2022] DELETE /identity/v3/users/b257de62315f4a03a6c0090a9156bfef => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:37.733189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.733189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.734047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.755128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93009870-2d34-4737-a7ac-80641a410ae7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.761119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:37.762697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:37.834606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.835656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.836070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.838202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.838202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.840428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.847992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:37.874890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1132/2295] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:37 2022] POST /identity/v3/users => generated 298 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:37.881703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.888408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:37.916598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1164/2296] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:37.922172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde9d9c-ba21-40cd-8c62-0cad73c5568f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.927420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:37.928605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:37.929210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:37.929738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:37.930571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:37.932804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.934793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:37.943649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:37.963677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1133/2297] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:37.966653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:37.990594 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efaf512-a340-464e-a633-34749b26390c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:37.999630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.001234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.002877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.003363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] PATH_INFO: `/v3/projects/82163bcc49ff431bba4ab1103b42fc13/users/36bea145cbe24d819d372b433084c8fb/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.033668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82163bcc49ff431bba4ab1103b42fc13, user_id=36bea145cbe24d819d372b433084c8fb, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.037129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.058090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ee8d0-6a9e-4068-a39e-d31fa6dd08d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.061077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1134/2298] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:37 2022] PUT /identity/v3/projects/82163bcc49ff431bba4ab1103b42fc13/users/36bea145cbe24d819d372b433084c8fb/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:38.062297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.083036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:38.147737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.148811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.149191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.149569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.150503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17834bc-22de-4e75-bdd3-97a583bad85b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.155526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.156222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.156512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.157015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.157767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.159465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.161099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.163284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcf6f78-2d83-4c4a-9602-51fa2cce76b9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.170324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1135/2299] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:38.175615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.176226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1165/2300] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:38.186624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a50f-7502-4dd6-89a9-2558285e48da admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.191767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.193602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.193909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.194188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] PATH_INFO: `/v3/projects/82163bcc49ff431bba4ab1103b42fc13/users/36bea145cbe24d819d372b433084c8fb/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.196401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=82163bcc49ff431bba4ab1103b42fc13, user_id=36bea145cbe24d819d372b433084c8fb, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.199289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.212573 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1136/2301] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:38 2022] PUT /identity/v3/projects/82163bcc49ff431bba4ab1103b42fc13/users/36bea145cbe24d819d372b433084c8fb/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:38.227839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.228428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.229424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.271978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] MFA Rules not processed for user `36bea145cbe24d819d372b433084c8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:38.273626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.364751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1166/2302] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:38.376109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.378359 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.454234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e733e96-5a3f-4ed4-95cf-9bb1fdefc4cc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.459106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.459776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.460120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.460859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.471733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.474152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.486667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1137/2303] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:38.520068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.521960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.557313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:38.623349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.625120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.640024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277f1d5-3ad6-4d84-90e2-6cf473875599 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.649685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.650356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.650640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.650915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.653778 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.657978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25422108-fb00-4619-806e-289bcd5b8930 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.665162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.666578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.666871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.667155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] PATH_INFO: `/v3/projects/73f8ae745d9d4d6b9391cc4cf54518db` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.668679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f8ae745d9d4d6b9391cc4cf54518db)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.670582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.732741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.734668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.745656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1138/2304] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:38.748299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f8ae745d9d4d6b9391cc4cf54518db', 'request_id': 'req-2aa28e3f-46bc-444d-b5fb-a6d07f987546'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.757054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.759249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.768793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f8ae745d9d4d6b9391cc4cf54518db', 'request_id': 'req-2aa28e3f-46bc-444d-b5fb-a6d07f987546'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.778810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506fc36c-6877-46e8-982c-707a8d6792ae service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.785077 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f8ae745d9d4d6b9391cc4cf54518db', 'request_id': 'req-2aa28e3f-46bc-444d-b5fb-a6d07f987546'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.803674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] The token cache is being invalidate because project 73f8ae745d9d4d6b9391cc4cf54518db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:38.804265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.806379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.810729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1167/2305] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:38 2022] DELETE /identity/v3/projects/73f8ae745d9d4d6b9391cc4cf54518db => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:38.812980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:38.814667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:38.842163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:38.880795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.882225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.882808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.883473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] PATH_INFO: `/v3/users/27cc29b393b44f20928eca7ac6c7bed9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.885434 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27cc29b393b44f20928eca7ac6c7bed9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.887978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.899708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa28e3f-46bc-444d-b5fb-a6d07f987546 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:38.904881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:38.906067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:38.906356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:38.906953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] PATH_INFO: `/v3/users/eae936169ef44209a09f897159a515ed` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:38.908200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae936169ef44209a09f897159a515ed)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:38.911352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:38.949717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27cc29b393b44f20928eca7ac6c7bed9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.952030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27cc29b393b44f20928eca7ac6c7bed9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.971770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27cc29b393b44f20928eca7ac6c7bed9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.972530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27cc29b393b44f20928eca7ac6c7bed9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.980043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae936169ef44209a09f897159a515ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.982789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695922f6-e128-40b1-8f4e-644bcba36043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27cc29b393b44f20928eca7ac6c7bed9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:38.984941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:38.989480 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae936169ef44209a09f897159a515ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.002388 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1139/2306] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:38 2022] DELETE /identity/v3/users/27cc29b393b44f20928eca7ac6c7bed9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:39.007773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.008235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.008546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.010062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae936169ef44209a09f897159a515ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.010947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae936169ef44209a09f897159a515ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.011583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae936169ef44209a09f897159a515ed'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.026755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:39.028023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.028909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1168/2307] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:38 2022] DELETE /identity/v3/users/eae936169ef44209a09f897159a515ed => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:39.069619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.071262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.110017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1140/2308] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:39.127260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.128726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.157757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196b554-faf6-4aaa-816f-d76c9d9f4a57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.166690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.167626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.168015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.168384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] PATH_INFO: `/v3/users/83344606769f4639a527880b8fb77f09` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.169669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=83344606769f4639a527880b8fb77f09)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.171643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.216455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a71fdd-41bf-41a3-a6b0-c729a9e1bb8b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.220591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.221356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.221652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.221934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.230048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83344606769f4639a527880b8fb77f09'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.232569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.234212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8310360-77a9-4287-b009-040add8ae6a2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.237418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83344606769f4639a527880b8fb77f09'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.247163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1141/2309] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:39.250429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83344606769f4639a527880b8fb77f09'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.251455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83344606769f4639a527880b8fb77f09'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.252314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83344606769f4639a527880b8fb77f09'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.273347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1169/2310] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:39 2022] DELETE /identity/v3/users/83344606769f4639a527880b8fb77f09 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:39.277995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.278365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.278726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.287613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:39.303701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:39.305755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.354062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.355663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.398356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1142/2311] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:39.409718 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.411925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.435883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:39.459534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21eefb5-5a54-499b-bf02-7ed7f4bcf17c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.464731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.465593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.465869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.466152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] PATH_INFO: `/v3/projects/df26de345652499ab776f7038707367d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.467392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df26de345652499ab776f7038707367d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.469778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.497247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604426d-87f2-4a33-b95c-b64d042233f1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.503563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.506254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.506539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.506971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.520944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.522526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.529474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df26de345652499ab776f7038707367d', 'request_id': 'req-ab656fd7-bb4e-4524-82c1-9808820c518f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.535832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df26de345652499ab776f7038707367d', 'request_id': 'req-ab656fd7-bb4e-4524-82c1-9808820c518f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.542294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df26de345652499ab776f7038707367d', 'request_id': 'req-ab656fd7-bb4e-4524-82c1-9808820c518f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.552634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1143/2312] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:57:39.560540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] The token cache is being invalidate because project df26de345652499ab776f7038707367d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:39.560540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.560540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.561616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1170/2313] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:39 2022] DELETE /identity/v3/projects/df26de345652499ab776f7038707367d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:39.566337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.580197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27975aa8-0a71-4181-8ed4-787de5fce20d service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.590918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:39.593293 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.668742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.670120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.670618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.671114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] PATH_INFO: `/v3/users/e62583e51a2f4895857848e83b690dc9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.673335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62583e51a2f4895857848e83b690dc9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.677285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.713531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab656fd7-bb4e-4524-82c1-9808820c518f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:39.719430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:39.720552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.720991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.721506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] PATH_INFO: `/v3/projects/d6c2b56305d240369d7b346ca7fbd26a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.722874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c2b56305d240369d7b346ca7fbd26a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:39.735508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:39.781737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62583e51a2f4895857848e83b690dc9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.782922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62583e51a2f4895857848e83b690dc9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.786775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:39.812152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62583e51a2f4895857848e83b690dc9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.813691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62583e51a2f4895857848e83b690dc9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.820258 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c2b56305d240369d7b346ca7fbd26a', 'request_id': 'req-f80943d7-3810-4226-84dc-da757e25093c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.829768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ce9d5cf9-7ae2-4562-b468-e262d698769d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62583e51a2f4895857848e83b690dc9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.831158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c2b56305d240369d7b346ca7fbd26a', 'request_id': 'req-f80943d7-3810-4226-84dc-da757e25093c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.841322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c2b56305d240369d7b346ca7fbd26a', 'request_id': 'req-f80943d7-3810-4226-84dc-da757e25093c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.852970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1144/2314] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:39 2022] DELETE /identity/v3/users/e62583e51a2f4895857848e83b690dc9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:39.862812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] The token cache is being invalidate because project d6c2b56305d240369d7b346ca7fbd26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:39.864187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.864684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f80943d7-3810-4226-84dc-da757e25093c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:39.866754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1171/2315] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:39 2022] DELETE /identity/v3/projects/d6c2b56305d240369d7b346ca7fbd26a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:39.871532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.871875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.872143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:39.892704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:39.893923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:39.973142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1145/2316] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:39.981244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:39.981535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:39.981872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.018555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:40.020653 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.021616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.023524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.036176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1172/2317] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:40.042447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1023b3-2886-4901-ae10-d9b312ec1fd7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.047412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.050898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.052258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.052852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.053322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] PATH_INFO: `/v3/projects/39e8c4e4dc8c4d28afb69497ad759e63` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.055083 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e8c4e4dc8c4d28afb69497ad759e63)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.057396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.064705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:40.085328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2347b-49ac-4f28-9a97-1e87fe0c4209 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.098531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.098531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.098531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.098531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.099427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:40.099427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.105892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.125991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e8c4e4dc8c4d28afb69497ad759e63', 'request_id': 'req-02dceb73-4621-4360-90f2-58594e1fd509'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.127143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1173/2318] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:40.132270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.142477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e8c4e4dc8c4d28afb69497ad759e63', 'request_id': 'req-02dceb73-4621-4360-90f2-58594e1fd509'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.143391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ba395-4ab5-48f2-a8a1-1934c7590bb4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.151636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e8c4e4dc8c4d28afb69497ad759e63', 'request_id': 'req-02dceb73-4621-4360-90f2-58594e1fd509'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.176961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] The token cache is being invalidate because project 39e8c4e4dc8c4d28afb69497ad759e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:40.177323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.177924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.182855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1146/2319] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:40 2022] DELETE /identity/v3/projects/39e8c4e4dc8c4d28afb69497ad759e63 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:40.185032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.189064 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.240641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.241542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.241734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.241969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.243239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.246700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.279475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:40.282360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1174/2320] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:40.283615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.285835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.299521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dceb73-4621-4360-90f2-58594e1fd509 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.362622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b01e09-970b-4ecf-aa44-9a0682ec18bc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.368191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.368465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.369081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.369370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.369477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.369667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] PATH_INFO: `/v3/users/36bea145cbe24d819d372b433084c8fb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.369760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.369977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] PATH_INFO: `/v3/projects/ccfdc19e632e4d11aac890d15b724538` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.371106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36bea145cbe24d819d372b433084c8fb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.371223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfdc19e632e4d11aac890d15b724538)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.373096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.373236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.428992 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36bea145cbe24d819d372b433084c8fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.429669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfdc19e632e4d11aac890d15b724538', 'request_id': 'req-a3174707-a039-4a4c-a376-8f821b15bd57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.429867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36bea145cbe24d819d372b433084c8fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.435741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfdc19e632e4d11aac890d15b724538', 'request_id': 'req-a3174707-a039-4a4c-a376-8f821b15bd57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.441407 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfdc19e632e4d11aac890d15b724538', 'request_id': 'req-a3174707-a039-4a4c-a376-8f821b15bd57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.442854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36bea145cbe24d819d372b433084c8fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.443681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36bea145cbe24d819d372b433084c8fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.451312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-66a50f61-4568-4890-a0e2-755f205416b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36bea145cbe24d819d372b433084c8fb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.454959 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] The token cache is being invalidate because project ccfdc19e632e4d11aac890d15b724538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:40.455277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.455967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.457783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1175/2321] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:40 2022] DELETE /identity/v3/projects/ccfdc19e632e4d11aac890d15b724538 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:40.458566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.459862 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.465370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1147/2322] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:40 2022] DELETE /identity/v3/users/36bea145cbe24d819d372b433084c8fb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:40.468374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:40.468483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535beb-5871-43bb-b75c-57bb083fca5a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.468483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535beb-5871-43bb-b75c-57bb083fca5a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.468483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82535beb-5871-43bb-b75c-57bb083fca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.489947 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-82535beb-5871-43bb-b75c-57bb083fca5a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:40.491235 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-82535beb-5871-43bb-b75c-57bb083fca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.556157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3174707-a039-4a4c-a376-8f821b15bd57 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.561553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.562261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.562551 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.562834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.563575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.565646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.568013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1148/2323] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:40.571674 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.572160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.572454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.588221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1176/2324] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:40.592130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.593194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:40.594400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.602067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e6399-c145-4497-baa7-7b8ef7f54e0f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.602382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1149/2325] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:40.606895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.609901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.610581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.610878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.611183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.611947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.613567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.628187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1177/2326] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:40.629114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.630788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a165c-52b3-4cc3-8e53-ae1a8730d5aa None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.639939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542c6fa-928c-41f4-b42e-07a2e7876cd3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.644786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.645826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.645826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.646033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.646846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.648478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.654535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.655228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.655707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.656040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.657340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:40.657662 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.659659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.659774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1178/2327] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:40.663720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:40.664397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.675339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7641cad-07ab-4bea-9cfb-9c1f8fa2ad0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.679515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1150/2328] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:57:40.683560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.683746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.684809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.685144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.685375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] PATH_INFO: `/v3/projects/7bb15f1d3e524d67b5493cb483ee8547/users/3acb774848a74e74a5886ee681ed205c/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.695066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09277f8c-fec5-4846-9bb9-17fd6ce38446 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.700415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.701079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.701371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.701660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.701969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bb15f1d3e524d67b5493cb483ee8547, user_id=3acb774848a74e74a5886ee681ed205c, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.702395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.704472 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.704684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b8be-ff70-411b-bb9c-c06090c196b6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.720366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1179/2329] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:40 2022] PUT /identity/v3/projects/7bb15f1d3e524d67b5493cb483ee8547/users/3acb774848a74e74a5886ee681ed205c/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:57:40.726239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.726519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.726813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.736107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1151/2330] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:40.741071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.745140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:40.746355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.829343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d669b-44fe-4904-abc2-83f448705bf4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.834219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.834646 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1180/2331] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:40.834901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.835193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.835474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.835567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.836215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.838129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.838998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.853011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6de7f-486b-49ab-9372-633d84b50917 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.858530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.859390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.859705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.860008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] PATH_INFO: `/v3/projects/82163bcc49ff431bba4ab1103b42fc13` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.863312 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1152/2332] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:40 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:40.863449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82163bcc49ff431bba4ab1103b42fc13)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:40.863565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.863565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.863808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:40.907776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:40.911616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82163bcc49ff431bba4ab1103b42fc13', 'request_id': 'req-44caf5e9-0d52-4b41-b5f4-1506f27572a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.923369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82163bcc49ff431bba4ab1103b42fc13', 'request_id': 'req-44caf5e9-0d52-4b41-b5f4-1506f27572a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.924470 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed59d304-db09-44bd-a7fb-d2f94b4b1095 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:40.940271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82163bcc49ff431bba4ab1103b42fc13', 'request_id': 'req-44caf5e9-0d52-4b41-b5f4-1506f27572a3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.957576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] The token cache is being invalidate because project 82163bcc49ff431bba4ab1103b42fc13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:40.958228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.960074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:40.961929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1181/2333] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:40 2022] DELETE /identity/v3/projects/82163bcc49ff431bba4ab1103b42fc13 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:40.962756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:40.964244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:40.979994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:40.980873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:40.981052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:40.981490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:40.983353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.028651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:41.049919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caf5e9-0d52-4b41-b5f4-1506f27572a3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.056513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.057200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.057491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.057771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.058519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.060183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc09b7f-81d7-4591-aa1e-fd0575d88e54 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.074721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1182/2334] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:41.074721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2db69-25f3-4044-8be7-65fac11716a6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.074721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2db69-25f3-4044-8be7-65fac11716a6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.074721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2db69-25f3-4044-8be7-65fac11716a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.077027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.079499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.092598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-51c2db69-25f3-4044-8be7-65fac11716a6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:41.093508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1153/2335] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:41.094130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-51c2db69-25f3-4044-8be7-65fac11716a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.094350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.103470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1183/2336] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:41.106018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0c472-90d8-4e82-b4ec-a8830da73e1b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.112121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.112705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.113051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.118411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.119463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.119940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.120213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.121149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.123926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.135507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:41.139483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1154/2337] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:41.139731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.140492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.145669 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1184/2338] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:41.149603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.154028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2379941-7d9f-471d-a1ed-1704c1d3f3dd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.158902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2fcbe-0d8f-492b-a02f-2f057fae3d27 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.159399 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.160368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.160880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.161193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] PATH_INFO: `/v3/projects/c5e2258f718e49fc966e425c77558b90/users/3bd8c4ec599143a0bfea05729ed4f4b7/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.164852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.165462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.165748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.166030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.166762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:41.167074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.168710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.186608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1185/2339] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 13:57:41.188270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e2258f718e49fc966e425c77558b90, user_id=3bd8c4ec599143a0bfea05729ed4f4b7, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.190842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.191303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c27412-9e7e-4e52-b5ac-05bcdd5d1f10 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.202363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cf4b25-d26c-4bc7-abe8-8e8c0adf9d76 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.208453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1155/2340] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:41 2022] PUT /identity/v3/projects/c5e2258f718e49fc966e425c77558b90/users/3bd8c4ec599143a0bfea05729ed4f4b7/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:41.219690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.220041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.220426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.263379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] MFA Rules not processed for user `3bd8c4ec599143a0bfea05729ed4f4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:41.264837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.278529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.279521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.281448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.281743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.282527 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.284624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.318853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1186/2341] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:41.324956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.360035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1156/2342] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:41.367843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:41.377627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.379622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.406732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6f3bc-2aeb-4279-ab3d-f3d6d4e3ef4f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.416147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.416894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.417180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.417458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.418175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.421558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.469231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d99d29-f41f-479d-b1b8-6557ad318348 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.469774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1187/2343] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/users => generated 287 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:41.474271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.494237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac5e77-5025-49d5-9d4b-e5dbe031a10a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.506607 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.507285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.507573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.509843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.511329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.514163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.524174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:41.533405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1188/2344] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:41.538519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.546401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.547135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.547421 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.547700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.559944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11c9e3-c475-4593-93ec-6153c6387ce3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.567359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.568162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.568506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.568793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.569536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.571207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.584898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1189/2345] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:41.591058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.602004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ccff06-f4bb-4c7f-b9ac-0211757bf8ac admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.607179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.608152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.608455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.608741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] PATH_INFO: `/v3/projects/b8708171d18747eda57bed0a3cb3030b/users/ca15829f779b403d8a78940b17dccf6a/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.617589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.619783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56501aec-4d89-4086-b2ec-eaa07e40641b service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.625821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8708171d18747eda57bed0a3cb3030b, user_id=ca15829f779b403d8a78940b17dccf6a, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.628807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.633128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1157/2346] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:41.651008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1190/2347] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:41 2022] PUT /identity/v3/projects/b8708171d18747eda57bed0a3cb3030b/users/ca15829f779b403d8a78940b17dccf6a/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:41.655949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a859-7f7a-47dd-8e62-7149f83f0523 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.656254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a859-7f7a-47dd-8e62-7149f83f0523 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.656497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905a859-7f7a-47dd-8e62-7149f83f0523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.675866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a905a859-7f7a-47dd-8e62-7149f83f0523 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:41.677574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a905a859-7f7a-47dd-8e62-7149f83f0523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.757461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1158/2348] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:41.767100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:41.768863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:41.848574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4079e12-878b-4c1f-8019-801a55bfb1c9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:41.853280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:41.854367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:41.854765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:41.855101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:41.874323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:41.875800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:41.903126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1191/2349] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:43.849023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b2c9-f88e-410a-8db1-366341ab1acb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:43.849670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b2c9-f88e-410a-8db1-366341ab1acb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:43.849670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e00b2c9-f88e-410a-8db1-366341ab1acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:43.899980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9e00b2c9-f88e-410a-8db1-366341ab1acb None None] MFA Rules not processed for user `ca15829f779b403d8a78940b17dccf6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:43.901504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9e00b2c9-f88e-410a-8db1-366341ab1acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:43.999536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1159/2350] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:44.011221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:44.012807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:44.156202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a1b8-6ce1-46dc-a659-d6149f2c646b service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:44.160352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:44.161411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:44.161853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:44.163727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:44.173972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:44.176675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5afb1-1605-445a-a770-55423e9ef7ec service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:44.200448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1192/2351] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:45.743387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:45.744161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:45.744161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:45.780225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] MFA Rules not processed for user `ca15829f779b403d8a78940b17dccf6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:45.781931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:45.792192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1160/2352] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:47.465572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.466140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.467033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.496281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] MFA Rules not processed for user `3acb774848a74e74a5886ee681ed205c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:47.497601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:47.507734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:47.576336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1193/2353] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:47.580762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.582530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:47.596155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83de3f4-268a-4619-9067-24322a6eec45 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.602188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.602861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.603331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.603616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.604352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.606249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:47.621970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:47.639402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1161/2354] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:47 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:47.643561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.656109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf16be-2b27-4766-8b08-8cc5eae27840 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.722122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.724742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.725032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.725309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.726016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.727981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:47.752785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1194/2355] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:47 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:47.757149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.770647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21bbf7-d6d4-4ae8-bfd1-ebddf71aaa5d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.775648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.776364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.776651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.776933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.777673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.779289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:47.797863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1162/2356] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:47.802977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.815537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a2548-d38a-4492-8056-5cb55624bb10 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.821751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.822712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.822949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.823232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] PATH_INFO: `/v3/projects/7acb9a364a7a40bbbaacdd321517b426/users/e0ef9587128f470ab550b38a4b7d4eb1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.846259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7acb9a364a7a40bbbaacdd321517b426, user_id=e0ef9587128f470ab550b38a4b7d4eb1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.849808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:47.867858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1195/2357] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:47 2022] PUT /identity/v3/projects/7acb9a364a7a40bbbaacdd321517b426/users/e0ef9587128f470ab550b38a4b7d4eb1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:47.871019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.885061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d2547-fafa-4872-9b95-1f0d33d99b01 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.953348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.954030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.954325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.954638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.955377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.957583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:47.970773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1163/2358] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:47.973555 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:47.985954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c73c7d-813e-40d4-b470-b433a72ee426 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:47.991749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:47.992772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:47.993126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:47.993493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] PATH_INFO: `/v3/projects/7acb9a364a7a40bbbaacdd321517b426/users/e0ef9587128f470ab550b38a4b7d4eb1/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:47.995800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7acb9a364a7a40bbbaacdd321517b426, user_id=e0ef9587128f470ab550b38a4b7d4eb1, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:47.998610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a59cbe-5d66-4268-b771-c93ede9cbf4f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:48.011723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1196/2359] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:47 2022] PUT /identity/v3/projects/7acb9a364a7a40bbbaacdd321517b426/users/e0ef9587128f470ab550b38a4b7d4eb1/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:48.946452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:48.949214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:48.963258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52bb85-cf8c-4e0c-88bd-7981f3da8cec admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:49.031503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:49.032441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:49.032737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:49.033016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] PATH_INFO: `/v3/users/e39d54b056dc4c2ca42c33e85391adf4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:49.034217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39d54b056dc4c2ca42c33e85391adf4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:49.036939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:49.115536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39d54b056dc4c2ca42c33e85391adf4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:49.116547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39d54b056dc4c2ca42c33e85391adf4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:49.133584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39d54b056dc4c2ca42c33e85391adf4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:49.134821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39d54b056dc4c2ca42c33e85391adf4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:49.142877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39d54b056dc4c2ca42c33e85391adf4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:49.155656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:49.165061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1164/2360] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:57:48 2022] DELETE /identity/v3/users/e39d54b056dc4c2ca42c33e85391adf4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:49.168103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:49.168603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:49.168844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:49.188108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:49.189321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:49.246448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:49.264363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1197/2361] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:49.276719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:49.278514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:49.359433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ed764-3f11-416c-b285-14c8d26a3979 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:49.364617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:49.365300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:49.365620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:49.365894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:49.377684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:49.379230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da4ad6-0f6a-4d48-b9b1-3ae12952fd7c service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:49.400331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1165/2362] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:50.070704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.072517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:50.098610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e44a8-bb32-422f-b961-831b355b2cdc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.103686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.104650 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.104944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.105233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] PATH_INFO: `/v3/projects/8e3df757b60e41009a03e11542ddf43c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.106461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3df757b60e41009a03e11542ddf43c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.108569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.174522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3df757b60e41009a03e11542ddf43c', 'request_id': 'req-92a176a9-b552-4851-b8d1-66a33b8a2904'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:50.182092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3df757b60e41009a03e11542ddf43c', 'request_id': 'req-92a176a9-b552-4851-b8d1-66a33b8a2904'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:50.188645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3df757b60e41009a03e11542ddf43c', 'request_id': 'req-92a176a9-b552-4851-b8d1-66a33b8a2904'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:50.203239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] The token cache is being invalidate because project 8e3df757b60e41009a03e11542ddf43c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:57:50.203853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:50.204548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-92a176a9-b552-4851-b8d1-66a33b8a2904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:57:50.206307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1198/2363] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:57:50 2022] DELETE /identity/v3/projects/8e3df757b60e41009a03e11542ddf43c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:50.212392 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.212990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.214820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.251625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:50.253353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:50.368671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1166/2364] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:57:50.378384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.379260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.379549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.403267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:50.404632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:50.417813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1199/2365] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:50.422228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.434742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849fe80-9721-4041-b8a2-39ce36a0d625 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.441147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.442150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.442635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.443108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.444997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:57:50.446805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.449049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.469562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1167/2366] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:57:50.476769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.499660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ddc17-b32d-4078-a8b8-4e2d8e53d621 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.512983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.513820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.514217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.515262 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.516136 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.518179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.554401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1200/2367] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:57:50 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:57:50.555228 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.567045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f55cf-658e-471b-a421-5d6eaceb672c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.658263 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.658978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.659327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.659853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.661666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.663492 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.703923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1168/2368] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:57:50 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:57:50.708471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.721424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d65a1-e2d0-4e47-b7ae-9d61c0146598 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.727272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.728347 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.728778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.729132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.730104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.732727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.748695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1201/2369] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:57:50.753499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.770264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b86a6-093b-4bb3-9615-c9b22633c6fa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.778310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.779153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.779517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.779942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.781173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.783428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372c6ae-5fb6-4452-9faf-d046318df529 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.791155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:50.804891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:50.805993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1169/2370] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:57:50.825512 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bdbde-fbfb-4292-9669-5732f47a6a23 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:50.840850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:50.841905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.842812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.842812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] PATH_INFO: `/v3/projects/805fc2a720ce4f52ba6f953154cb2bf0/users/89d9658e21cb42c6b4e867ab37b2d9e3/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:50.917678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805fc2a720ce4f52ba6f953154cb2bf0, user_id=89d9658e21cb42c6b4e867ab37b2d9e3, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:50.934361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:50.963627 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1202/2371] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:57:50 2022] PUT /identity/v3/projects/805fc2a720ce4f52ba6f953154cb2bf0/users/89d9658e21cb42c6b4e867ab37b2d9e3/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:57:50.975737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4da22d-352f-44e7-bbb4-2741538aa220 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:50.976134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4da22d-352f-44e7-bbb4-2741538aa220 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:50.976498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4da22d-352f-44e7-bbb4-2741538aa220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:51.010614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4d4da22d-352f-44e7-bbb4-2741538aa220 None None] MFA Rules not processed for user `89d9658e21cb42c6b4e867ab37b2d9e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:51.012191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4d4da22d-352f-44e7-bbb4-2741538aa220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:51.184361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1170/2372] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:51.193586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:51.196011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:51.319099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a9b5a-9ca7-4a18-906b-76943d316756 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:51.336356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:51.337361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:51.337868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:51.338251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:51.362711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:51.366711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:51.408132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1203/2373] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:51 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:57:56.846560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9bd8f-5fe4-47c6-8791-555f4f7e51a2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:56.847686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9bd8f-5fe4-47c6-8791-555f4f7e51a2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:56.847686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9bd8f-5fe4-47c6-8791-555f4f7e51a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:56.877244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-19d9bd8f-5fe4-47c6-8791-555f4f7e51a2 None None] MFA Rules not processed for user `e0ef9587128f470ab550b38a4b7d4eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:56.880275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-19d9bd8f-5fe4-47c6-8791-555f4f7e51a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:56.964539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1171/2374] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:56.980174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:57:56.982473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:57.016845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:57:57.120061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c66f-251e-48b4-9182-a1775f8e07fe None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:57.120697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c66f-251e-48b4-9182-a1775f8e07fe None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:57.121158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886c66f-251e-48b4-9182-a1775f8e07fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:57.160918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d886c66f-251e-48b4-9182-a1775f8e07fe None None] MFA Rules not processed for user `532b36f272dc432aad6faa294bd1bc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:57:57.162382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d886c66f-251e-48b4-9182-a1775f8e07fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:57.190035 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6859-3b4d-4675-99a8-0be72ff250d3 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:57:57.197546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:57:57.198530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:57:57.198982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:57:57.200157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:57:57.202514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:57:57.280526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1172/2375] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:57:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:57:57.303185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:57:57.304113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:57:57.315964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1204/2376] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:57:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:58:07.247893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:07.309297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=59 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:11.338369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f3be9-4eb5-42a8-9a3b-dc1f4efcc992 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:11.338829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f3be9-4eb5-42a8-9a3b-dc1f4efcc992 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:11.339037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f3be9-4eb5-42a8-9a3b-dc1f4efcc992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:11.374271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4e3f3be9-4eb5-42a8-9a3b-dc1f4efcc992 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:11.375485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4e3f3be9-4eb5-42a8-9a3b-dc1f4efcc992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:11.451978 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1173/2377] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:58:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:11.495963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:11.497419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:11.567978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed4e65-8568-4059-9d42-9f6d25eadf1e service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:11.572916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:11.573560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:11.573842 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:11.574120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:11.581983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:11.584588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:11.592397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1205/2378] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:58:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:58:12.113972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7c07-4d1a-4871-8813-3867d81875a6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.114411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7c07-4d1a-4871-8813-3867d81875a6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.114907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f7c07-4d1a-4871-8813-3867d81875a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.136348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0c8f7c07-4d1a-4871-8813-3867d81875a6 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:12.137689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0c8f7c07-4d1a-4871-8813-3867d81875a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:12.147864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1174/2379] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 13:58:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:58:12.161503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:12.172111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d1a45-162b-4437-a1b8-00bef03cb756 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:12.178210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:12.179343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.179806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.180273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.191569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:12.193274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:12.201870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1206/2380] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:58:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 13:58:12.420464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10173ef-5ff8-4e1b-9e49-131decda5513 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.420929 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10173ef-5ff8-4e1b-9e49-131decda5513 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.421032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10173ef-5ff8-4e1b-9e49-131decda5513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.440024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b10173ef-5ff8-4e1b-9e49-131decda5513 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:12.441358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b10173ef-5ff8-4e1b-9e49-131decda5513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:12.450546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1175/2381] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:58:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:12.465059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:12.477829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e7087-d8e2-4c87-b1d2-b9f265e1e502 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:12.483885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:12.484683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.485045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.485401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.494954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:12.503642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:12.507110 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1207/2382] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:58:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 13:58:12.840164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.840463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.840749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.858913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:12.860176 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:12.868963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1176/2383] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 13:58:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:12.880836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:12.891270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942145d-fa79-4178-bbfe-18a8bf753cea service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:12.896357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:12.896992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:12.897311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:12.897596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:12.905508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:12.907093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc99912c-9e10-4b6a-87e1-a81332a49348 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:12.915631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1208/2384] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:58:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:58:15.021038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.022740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.088882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d626dd-b40b-49df-aa98-14abffa38d32 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.093471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.094349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.094639 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.094922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] PATH_INFO: `/v3/users/60bbd2f0609b4b8d81c9db106408eba8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.096167 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60bbd2f0609b4b8d81c9db106408eba8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.098091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.143125 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60bbd2f0609b4b8d81c9db106408eba8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.143777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60bbd2f0609b4b8d81c9db106408eba8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.155568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60bbd2f0609b4b8d81c9db106408eba8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.156246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60bbd2f0609b4b8d81c9db106408eba8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.163600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60bbd2f0609b4b8d81c9db106408eba8'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.167130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:15.177430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1177/2385] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:58:15 2022] DELETE /identity/v3/users/60bbd2f0609b4b8d81c9db106408eba8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:15.182074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.182624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.182624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.200141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:15.201340 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.258449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1209/2386] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:58:15.268272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.269621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.328552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02e83c-f065-49af-bb41-d11dbf24e00d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.332145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.332788 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.333971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.334078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.343343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.344801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427af45-70a1-4d0d-a99d-1a31e62072d8 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.354070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1178/2387] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:15.518383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.519771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.533458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc29d38-014e-4d62-b4c1-f37cf8bcbc61 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.538271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.539174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.539462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.539742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] PATH_INFO: `/v3/projects/456ea6bbe702420cb7fa4284088efda2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.540988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=456ea6bbe702420cb7fa4284088efda2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.542920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.587859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:15.590919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '456ea6bbe702420cb7fa4284088efda2', 'request_id': 'req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.598449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '456ea6bbe702420cb7fa4284088efda2', 'request_id': 'req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.604482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '456ea6bbe702420cb7fa4284088efda2', 'request_id': 'req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.617177 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] The token cache is being invalidate because project 456ea6bbe702420cb7fa4284088efda2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:58:15.617478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.618097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-912486db-6e7c-46a3-8b66-8ba08a6a8c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:15.619879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1210/2388] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:58:15 2022] DELETE /identity/v3/projects/456ea6bbe702420cb7fa4284088efda2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:15.624727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.625026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.625307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.645258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:15.646528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.715857 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1179/2389] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:15.723979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.724254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.724541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.752001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:15.753208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:15.762497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1211/2390] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:15.766766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.781906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0404c0b4-ff2a-4631-9de4-bcb83c270af7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.790316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.790666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.791333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.791333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.792048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:58:15.792367 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.793972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.809117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1180/2391] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:58:15.815853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.824720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c43c0ea-f7da-4d60-8e34-b3dbd1ca9521 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.831762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.832727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.833635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.834181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.835163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.837141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.871698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1212/2392] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:58:15 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:58:15.877291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:15.889565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff99a2-e0d2-43af-8a7f-33a210ea1658 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:15.953703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:15.954379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:15.954721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:15.955053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:15.955739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:15.957936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:15.991199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1181/2393] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:58:15 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:58:15.994603 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:16.006836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cf174-f68c-40da-bc68-a5b896fbd0be admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:16.012747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:16.013428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:16.013737 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:16.014061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:16.014721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:16.016362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:16.027496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1213/2394] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:58:16.031455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:16.042714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475ef62-bd49-4873-81bf-0f24109e4c48 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:16.047271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:16.047967 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:16.048285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:16.048568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:16.049300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:16.050889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde48b11-3130-4fdb-a75f-d453cb693d4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:16.062123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1182/2395] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 13:58:16.066273 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:16.076285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428a0d0-6b7f-46bc-aeb4-359864d77f74 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:16.080855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:16.081743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:16.082028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:16.082307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] PATH_INFO: `/v3/projects/9a4447d3b8f744f5b1f5ba17ec3be70b/users/7bd54a57f3ef475ebe91b435970755fd/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:16.097393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4447d3b8f744f5b1f5ba17ec3be70b, user_id=7bd54a57f3ef475ebe91b435970755fd, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:16.100637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:16.114195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1214/2396] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:58:16 2022] PUT /identity/v3/projects/9a4447d3b8f744f5b1f5ba17ec3be70b/users/7bd54a57f3ef475ebe91b435970755fd/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:16.125456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:16.125752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:16.126292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:16.133831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:16.155876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] MFA Rules not processed for user `7bd54a57f3ef475ebe91b435970755fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:16.157179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:16.227968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1183/2397] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:16.239149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:16.241060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:16.326009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63abdf93-2162-4552-9b08-1e667e2331af admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:16.329613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:16.330254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:16.330541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:16.330818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:16.339391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:16.340885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9499-d7c9-43ae-bcb6-849667ef494c service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:16.350364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1215/2398] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:26.224829 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=30 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:26.345564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=4,keystone:SELECT=69,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:36.107046 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:36.108473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:36.199557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35fd3b-d455-43bb-b4e0-9515770d523f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:36.204346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:36.204988 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:36.205274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:36.205552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:36.207358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:36.218910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:36.220538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:36.230600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1184/2399] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:38.172325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a4f56-c3bf-4965-a442-3a9d4d092a29 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:38.172882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a4f56-c3bf-4965-a442-3a9d4d092a29 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:38.172882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a4f56-c3bf-4965-a442-3a9d4d092a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:38.194740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-619a4f56-c3bf-4965-a442-3a9d4d092a29 None None] MFA Rules not processed for user `ca15829f779b403d8a78940b17dccf6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:38.196046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-619a4f56-c3bf-4965-a442-3a9d4d092a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:38.261508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1216/2400] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:39.888710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:39.961128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5a95d-535a-42aa-955b-5dd3efed95c2 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:39.970137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:39.971405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:39.971686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:39.972021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] PATH_INFO: `/v3/users/7bd54a57f3ef475ebe91b435970755fd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:39.973317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd54a57f3ef475ebe91b435970755fd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:39.975297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:40.022652 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd54a57f3ef475ebe91b435970755fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.024097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd54a57f3ef475ebe91b435970755fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.040281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd54a57f3ef475ebe91b435970755fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.041346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd54a57f3ef475ebe91b435970755fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.048319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:40.050320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd54a57f3ef475ebe91b435970755fd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.064109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1185/2401] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:58:39 2022] DELETE /identity/v3/users/7bd54a57f3ef475ebe91b435970755fd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:58:40.068583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:40.069073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:40.069232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:40.087377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:40.088618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:40.150453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1217/2402] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:40.160416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:40.162069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:40.224577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d01da-1ad3-47b9-9acc-719706359851 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:40.228453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:40.229108 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:40.229402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:40.229685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:40.238561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:40.240016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905990e-45ec-4525-9a79-e060340e46cf service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:40.250310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1186/2403] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:40.784433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:40.786105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:40.802458 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde68fe7-93b3-4149-8c25-448843166417 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:40.809511 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:40.810604 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:40.810746 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:40.811055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:40.811343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:40.811771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:40.812455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] PATH_INFO: `/v3/projects/9a4447d3b8f744f5b1f5ba17ec3be70b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:40.814540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4447d3b8f744f5b1f5ba17ec3be70b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:40.818250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:40.830770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] MFA Rules not processed for user `3acb774848a74e74a5886ee681ed205c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:40.832031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:40.903615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4447d3b8f744f5b1f5ba17ec3be70b', 'request_id': 'req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.907974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1187/2404] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 13:58:40.911481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4447d3b8f744f5b1f5ba17ec3be70b', 'request_id': 'req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.921072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4447d3b8f744f5b1f5ba17ec3be70b', 'request_id': 'req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.938119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] The token cache is being invalidate because project 9a4447d3b8f744f5b1f5ba17ec3be70b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:58:40.938409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.939317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a2bba7eb-b8cb-4653-996b-d3bbe0e3aa99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:40.941318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1218/2405] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:58:40 2022] DELETE /identity/v3/projects/9a4447d3b8f744f5b1f5ba17ec3be70b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:58:43.005627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:43.007168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:43.025837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14011d-28f6-4d70-bad5-6e6baa662949 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:43.084285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:43.085195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:43.085487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:43.085771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] PATH_INFO: `/v3/users/3bd8c4ec599143a0bfea05729ed4f4b7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:43.086956 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd8c4ec599143a0bfea05729ed4f4b7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:43.088927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:43.129785 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:43.136528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd8c4ec599143a0bfea05729ed4f4b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.137290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd8c4ec599143a0bfea05729ed4f4b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.149160 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd8c4ec599143a0bfea05729ed4f4b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.149740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd8c4ec599143a0bfea05729ed4f4b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.156401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd8c4ec599143a0bfea05729ed4f4b7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.169348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1188/2406] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:58:43 2022] DELETE /identity/v3/users/3bd8c4ec599143a0bfea05729ed4f4b7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:43.174800 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:43.175372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:43.175372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:43.195341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:43.196615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:43.223860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:43.263625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1219/2407] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:43.273333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:43.274916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:43.333563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59a87-9288-4002-a34a-0769aca0ab27 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:43.337283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:43.337932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:43.338222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:43.338500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:43.347277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:43.348787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:43.358591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1189/2408] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:43.638525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:43.639982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:43.653060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a334c-1c99-4e96-9ab1-3baf905dde11 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:43.658096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:43.658966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:43.659257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:43.659537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] PATH_INFO: `/v3/projects/c5e2258f718e49fc966e425c77558b90` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:43.660798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e2258f718e49fc966e425c77558b90)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:43.662714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:43.708757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e2258f718e49fc966e425c77558b90', 'request_id': 'req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.715449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e2258f718e49fc966e425c77558b90', 'request_id': 'req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.721307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e2258f718e49fc966e425c77558b90', 'request_id': 'req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.733276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] The token cache is being invalidate because project c5e2258f718e49fc966e425c77558b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:58:43.733569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.734189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f4e8c1ea-01e5-438e-a82b-5cabc931bc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:43.735732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1220/2409] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:58:43 2022] DELETE /identity/v3/projects/c5e2258f718e49fc966e425c77558b90 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:53.354541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:53.730324 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:56.186760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:56.188187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:56.201168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef2c94-e3dd-4c7a-b7a6-73b78f7f0cc9 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:56.258641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:56.259506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:56.259796 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:56.260112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] PATH_INFO: `/v3/users/89d9658e21cb42c6b4e867ab37b2d9e3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:56.261301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=89d9658e21cb42c6b4e867ab37b2d9e3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:56.263215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:56.308533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89d9658e21cb42c6b4e867ab37b2d9e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.309184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89d9658e21cb42c6b4e867ab37b2d9e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.321213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89d9658e21cb42c6b4e867ab37b2d9e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.321917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89d9658e21cb42c6b4e867ab37b2d9e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.328744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89d9658e21cb42c6b4e867ab37b2d9e3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.341774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1190/2410] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:58:56 2022] DELETE /identity/v3/users/89d9658e21cb42c6b4e867ab37b2d9e3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:56.346300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:56.346754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:56.346847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:56.364629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:56.365809 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:56.423057 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1221/2411] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:56.432743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:56.434182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:56.491846 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78fae-226a-46c9-85ba-7dd238ff38ab admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:56.495543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:56.496244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:56.496537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:56.496816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:56.507622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:56.509865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404642b-ebb4-4b48-87a7-e57f306ad72d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:56.519379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1191/2412] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:58:56.836493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:56.837911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:56.849816 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87761dd-0654-401c-80f3-79c51d7b5731 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:56.855049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:56.855907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:56.856197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:56.856474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] PATH_INFO: `/v3/projects/805fc2a720ce4f52ba6f953154cb2bf0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:56.857681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805fc2a720ce4f52ba6f953154cb2bf0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:56.859602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:56.905587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805fc2a720ce4f52ba6f953154cb2bf0', 'request_id': 'req-c13fe035-a43c-4972-9290-5c60a76d2659'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.911520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805fc2a720ce4f52ba6f953154cb2bf0', 'request_id': 'req-c13fe035-a43c-4972-9290-5c60a76d2659'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.917643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805fc2a720ce4f52ba6f953154cb2bf0', 'request_id': 'req-c13fe035-a43c-4972-9290-5c60a76d2659'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.931186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] The token cache is being invalidate because project 805fc2a720ce4f52ba6f953154cb2bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:58:56.931546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.932215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c13fe035-a43c-4972-9290-5c60a76d2659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:58:56.933770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1222/2413] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:58:56 2022] DELETE /identity/v3/projects/805fc2a720ce4f52ba6f953154cb2bf0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:58:56.938615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:56.938927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:56.939215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:56.957044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:56.958213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:56.976330 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:57.032040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1192/2414] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:57.033705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.034007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.034295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.054537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:57.055764 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:57.064054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1223/2415] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:57.068215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.078036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2596ed6f-438d-4012-9862-addd1bcf68bf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.082484 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.083135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.083425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.083700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.084461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:58:57.084767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.086861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.098710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1193/2416] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:58:57.103993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.113171 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f408ac-5bb0-4633-9973-e8c0c4aabe9e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.119696 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.120406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.120695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.120977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.121678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.123594 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.152307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1224/2417] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:58:57 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:58:57.156389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.166871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea880-c1e5-408f-8b01-f5dd04ba13f0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.223565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.224225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.224463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.224745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.225445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.227635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.249647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1194/2418] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:58:57 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 13:58:57.254122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.264387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66657f6-9f58-479b-a300-be2e311511aa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.269616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.270280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.270569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.270848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.271579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.274114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.285245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1225/2419] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:58:57.289236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.300204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60078e5-38be-4144-a89c-88326ab23895 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.305174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.305858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.306150 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.306438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.307212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.308856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1341ae8f-7372-474c-a336-3db36cc08a81 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.320226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1195/2420] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:58:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:58:57.324368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.340860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e1241-2018-4193-9508-56e9fcafc921 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.346468 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.347510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.347802 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.348131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] PATH_INFO: `/v3/projects/b7615ff783054a67a0d7f89fc7938afb/users/5057292ce6224867b44f1d6ba6a9e3ec/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.364573 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7615ff783054a67a0d7f89fc7938afb, user_id=5057292ce6224867b44f1d6ba6a9e3ec, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.367525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.385980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1226/2421] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:58:57 2022] PUT /identity/v3/projects/b7615ff783054a67a0d7f89fc7938afb/users/5057292ce6224867b44f1d6ba6a9e3ec/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 13:58:57.396364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f88d-b7a2-4970-bef5-92dded6d2b2f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.396750 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f88d-b7a2-4970-bef5-92dded6d2b2f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.397172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19f88d-b7a2-4970-bef5-92dded6d2b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.426838 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-af19f88d-b7a2-4970-bef5-92dded6d2b2f None None] MFA Rules not processed for user `5057292ce6224867b44f1d6ba6a9e3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:58:57.428373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-af19f88d-b7a2-4970-bef5-92dded6d2b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:57.492528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1196/2422] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:58:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:58:57.503418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:58:57.505138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:58:57.527979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:58:57.563845 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ee287-b4e9-456a-a515-ed27a6710d8e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:58:57.567435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:58:57.568113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:58:57.568403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:58:57.568681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:58:57.577577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:58:57.579009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:58:57.588338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1227/2423] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:58:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:59:07.489619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:07.583744 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=17 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:16.940081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:16.940611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:16.940707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:16.989052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:16.990252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:17.052889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1197/2424] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 13:59:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:17.097993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:17.099396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:17.189454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b52053-8618-454e-a733-c69f1c2f54dc service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:17.195125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:17.195904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:17.196195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:17.196473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:17.204457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:17.206038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:17.214156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1228/2425] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:59:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:59:18.056207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:18.057645 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:18.131364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14290f8-39d0-46cd-b827-c4464da11d9c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:18.136070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:18.136881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:18.137200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:18.137403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:18.139208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:18.215245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:18.216723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da894e3-5786-4f95-a4c2-a7216aedeb3d service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:18.228081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1198/2426] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:59:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:59:27.211253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=41 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:28.222757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:28.403450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:28.520206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44908b70-90cf-44e3-8547-67ea356d4de2 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:28.526606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:28.527784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:28.527784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:28.528615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] PATH_INFO: `/v3/users/5057292ce6224867b44f1d6ba6a9e3ec` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:28.529564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=5057292ce6224867b44f1d6ba6a9e3ec)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:28.531476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:28.587909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5057292ce6224867b44f1d6ba6a9e3ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:28.595291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5057292ce6224867b44f1d6ba6a9e3ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:28.612850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5057292ce6224867b44f1d6ba6a9e3ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:28.613673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5057292ce6224867b44f1d6ba6a9e3ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:28.614350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5057292ce6224867b44f1d6ba6a9e3ec'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:28.634166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1229/2427] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:59:28 2022] DELETE /identity/v3/users/5057292ce6224867b44f1d6ba6a9e3ec => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:28.638783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:28.639090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:28.639381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:28.658152 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:28.659426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:28.735155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1199/2428] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:28.745514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:28.746952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:28.814994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec1e52-0933-44bd-af1d-1f9abfa328ef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:28.820490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:28.821161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:28.821455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:28.821742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:28.831209 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:28.832687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77812e2f-9c89-47ee-9db4-0815df6921a4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:28.843392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1230/2429] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 13:59:29.152856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:29.154687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:29.175275 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165f204-a7c7-4c92-876d-cec967122316 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:29.193464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:29.194976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:29.195260 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:29.195544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] PATH_INFO: `/v3/projects/b7615ff783054a67a0d7f89fc7938afb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:29.196798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7615ff783054a67a0d7f89fc7938afb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:29.198748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:29.256910 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7615ff783054a67a0d7f89fc7938afb', 'request_id': 'req-6c6613cc-6f60-4722-b092-310f8c992be7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:29.263615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7615ff783054a67a0d7f89fc7938afb', 'request_id': 'req-6c6613cc-6f60-4722-b092-310f8c992be7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:29.271245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7615ff783054a67a0d7f89fc7938afb', 'request_id': 'req-6c6613cc-6f60-4722-b092-310f8c992be7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:29.286271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] The token cache is being invalidate because project b7615ff783054a67a0d7f89fc7938afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:59:29.287124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:29.288136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:29.290128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1200/2430] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:59:29 2022] DELETE /identity/v3/projects/b7615ff783054a67a0d7f89fc7938afb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:33.536519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:33.537028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:33.537299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:33.554256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:33.555445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:33.575928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:33.614260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1231/2431] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 13:59:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:33.626802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:33.628241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:33.697570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6613cc-6f60-4722-b092-310f8c992be7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:33.702370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123345-244f-4383-8a21-9324a44762da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:33.703017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123345-244f-4383-8a21-9324a44762da service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:33.703305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123345-244f-4383-8a21-9324a44762da service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:33.703582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9123345-244f-4383-8a21-9324a44762da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:33.712403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123345-244f-4383-8a21-9324a44762da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:33.713979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9123345-244f-4383-8a21-9324a44762da service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:33.721801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1201/2432] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 13:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 13:59:35.844352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:35.844352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:35.912923 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bf640-50c9-437d-9a6f-d91c8e4a78c7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:35.917471 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:35.918341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:35.918628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:35.918910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] PATH_INFO: `/v3/users/ca15829f779b403d8a78940b17dccf6a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:35.920140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca15829f779b403d8a78940b17dccf6a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:35.922112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:35.967554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca15829f779b403d8a78940b17dccf6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:35.974784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca15829f779b403d8a78940b17dccf6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:35.986402 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca15829f779b403d8a78940b17dccf6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:35.987069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca15829f779b403d8a78940b17dccf6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:35.987671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fb938c0b-4fa3-4419-8af9-5557e43f14a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca15829f779b403d8a78940b17dccf6a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.000868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1232/2433] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:59:35 2022] DELETE /identity/v3/users/ca15829f779b403d8a78940b17dccf6a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:36.173723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123345-244f-4383-8a21-9324a44762da service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.185443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9123345-244f-4383-8a21-9324a44762da service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.196505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:36.245234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.246084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.246370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.246648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] PATH_INFO: `/v3/projects/b8708171d18747eda57bed0a3cb3030b` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.247855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8708171d18747eda57bed0a3cb3030b)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.250017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.294986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8708171d18747eda57bed0a3cb3030b', 'request_id': 'req-3e6a9f5b-4baa-42f1-a385-e8a27130116b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.300825 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8708171d18747eda57bed0a3cb3030b', 'request_id': 'req-3e6a9f5b-4baa-42f1-a385-e8a27130116b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.307007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8708171d18747eda57bed0a3cb3030b', 'request_id': 'req-3e6a9f5b-4baa-42f1-a385-e8a27130116b'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.319589 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] The token cache is being invalidate because project b8708171d18747eda57bed0a3cb3030b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:59:36.319908 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.320528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3e6a9f5b-4baa-42f1-a385-e8a27130116b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:36.322035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1202/2434] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:59:36 2022] DELETE /identity/v3/projects/b8708171d18747eda57bed0a3cb3030b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:36.327713 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.327942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.328206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.346095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:36.347280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:36.390878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:36.409761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1233/2435] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:36.418314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.418635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.418917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.437784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:36.438986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:36.447430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1203/2436] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:36.451639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.461484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050f8230-2f13-4f42-97e2-48ea31620087 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.466062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.466717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.467008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.467291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.468055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 13:59:36.468368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.469983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.481866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1234/2437] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 13:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 13:59:36.486035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.496252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa814f57-1e5e-4878-bbe2-20530703386f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.501319 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.501990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.502281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.502565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.503267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.505301 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.531119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1204/2438] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 13:59:36 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:59:36.535249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.545635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114a0fa-7cc1-42b4-8f9d-2a0f6e4cdd0e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.600437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.601161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.601453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.601736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.602432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.604376 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.623384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1235/2439] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 13:59:36 2022] POST /identity/v3/users => generated 286 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 13:59:36.627657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.638615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df5795-04f8-4438-bd95-798c914e7439 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.643190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.643856 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.644146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.644422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.645147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.647685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.658331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1205/2440] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:59:36.662529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.672553 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4aeb34-44b3-4bbd-bdd7-c50a63ac8add admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.677532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.678194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.678500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.678782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.679514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.681137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e336b0a-92f3-492f-8f83-aa10e06b925a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.690986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1236/2441] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 13:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 13:59:36.696183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.706011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d95dce-8c31-4eff-bd34-9652acb37bc8 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.711368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.712289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.712584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.712868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] PATH_INFO: `/v3/projects/8ea6873c3ec7400597d439376a7e5101/users/a58638da14c94c298b49688ff336289a/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.728198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea6873c3ec7400597d439376a7e5101, user_id=a58638da14c94c298b49688ff336289a, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.730884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.744603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1206/2442] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 13:59:36 2022] PUT /identity/v3/projects/8ea6873c3ec7400597d439376a7e5101/users/a58638da14c94c298b49688ff336289a/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:36.755672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.756086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.756375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.782708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] MFA Rules not processed for user `a58638da14c94c298b49688ff336289a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:36.784012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:36.852979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1237/2443] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:36.863966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:36.866025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:36.914317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:36.930341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c845e8-5825-4905-ba24-e0e7715a4de9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:36.934193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:36.934994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:36.935286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:36.935567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:36.945605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:36.947179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e1d1e-3434-43aa-8e8a-3a81a428109f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:36.956717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1207/2444] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:59:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:59:36.981451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:37.025772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:37.045386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212dc382-828e-41d3-bc1f-19e0f3bce7d3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:37.050023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:37.050891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:37.051182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:37.051477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] PATH_INFO: `/v3/users/a58638da14c94c298b49688ff336289a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:37.053017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58638da14c94c298b49688ff336289a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:37.054957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:37.103564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58638da14c94c298b49688ff336289a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.110243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58638da14c94c298b49688ff336289a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.122418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58638da14c94c298b49688ff336289a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.123128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58638da14c94c298b49688ff336289a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.123755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58638da14c94c298b49688ff336289a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.137644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1238/2445] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 13:59:36 2022] DELETE /identity/v3/users/a58638da14c94c298b49688ff336289a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:37.142135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:37.142432 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:37.142736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:37.160448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 13:59:37.162170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:37.224619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1208/2446] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 13:59:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 13:59:37.234264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:37.235771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:37.293729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e391b3-4f3b-4d2d-8f6c-83fb8284e6c9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:37.297461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:37.298117 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:37.298408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:37.298689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:37.308183 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:37.309636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94ccdc-d95a-4a6c-939f-b5ca214ba852 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:37.319753 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1239/2447] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 13:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 13:59:37.590239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 13:59:37.591690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 13:59:37.604093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8abb0-49de-4eae-9c9a-6ae2c5efa1b5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 13:59:37.610517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 13:59:37.611390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 13:59:37.611683 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 13:59:37.612001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] PATH_INFO: `/v3/projects/8ea6873c3ec7400597d439376a7e5101` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 13:59:37.613205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea6873c3ec7400597d439376a7e5101)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 13:59:37.615131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 13:59:37.665404 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea6873c3ec7400597d439376a7e5101', 'request_id': 'req-b6ef776d-c223-400a-a9a6-df36e0bcbee5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.671776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea6873c3ec7400597d439376a7e5101', 'request_id': 'req-b6ef776d-c223-400a-a9a6-df36e0bcbee5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.677431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea6873c3ec7400597d439376a7e5101', 'request_id': 'req-b6ef776d-c223-400a-a9a6-df36e0bcbee5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.690114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] The token cache is being invalidate because project 8ea6873c3ec7400597d439376a7e5101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 13:59:37.690411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.691035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b6ef776d-c223-400a-a9a6-df36e0bcbee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 13:59:37.692624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1209/2448] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 13:59:37 2022] DELETE /identity/v3/projects/8ea6873c3ec7400597d439376a7e5101 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 13:59:47.314866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 13:59:47.687066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:00:07.732856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:07.733392 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:07.733523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:07.750839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] MFA Rules not processed for user `3eb0459bdac1456a900046e264835a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:07.752066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:07.805812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1240/2449] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:00:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:00:17.798735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:00:34.259230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:34.259751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:34.259877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:34.278006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:34.279209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:34.337072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1210/2450] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:00:34.392018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:34.393442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:34.464655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845df93c-4a96-4830-a667-1a43b07fafb7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:34.469660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:34.470316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:34.470608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:34.470893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:34.479053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:34.480733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:34.489264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1241/2451] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:00:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:00:35.258031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:35.259771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:35.324862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593ab8-7fc2-4720-af12-309e5bc4f35e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:35.328670 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:35.329345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:35.329637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:35.330085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:35.333705 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:35.412957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:35.414744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12800c-f40e-48c6-bf4d-9669a0f72e91 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:35.430079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1211/2452] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:36.628976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:36.630537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:36.642728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afb99-6da7-4f57-a176-0e356cadb2f4 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:36.646749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:36.647405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:36.647697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:36.648133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:36.650632 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:36.665061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:36.666834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cdd4d4-4061-4a3b-90a9-17844f28f69b service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:36.683180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1242/2453] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:45.423246 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:00:46.674712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=38 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:00:49.260860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.261517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.262090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.263883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1212/2454] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:00:49.269941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60888f54-ae26-4913-b4ca-4ae5a2ab4d3e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.270408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60888f54-ae26-4913-b4ca-4ae5a2ab4d3e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.270494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60888f54-ae26-4913-b4ca-4ae5a2ab4d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.308245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-60888f54-ae26-4913-b4ca-4ae5a2ab4d3e None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:49.309501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-60888f54-ae26-4913-b4ca-4ae5a2ab4d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.318863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1243/2455] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:00:49.331605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:49.333721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.422021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ecc87-d83d-4fe6-b8c9-ad4b25c35555 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:49.425759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01fb885-99e4-4028-8035-855057466321 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:49.426407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb885-99e4-4028-8035-855057466321 service glance] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.426692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb885-99e4-4028-8035-855057466321 service glance] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.426973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01fb885-99e4-4028-8035-855057466321 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.436940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fb885-99e4-4028-8035-855057466321 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:49.438365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01fb885-99e4-4028-8035-855057466321 service glance] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:49.449298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1213/2456] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:49.516919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6429f-37bc-49de-98d8-3b47dc4274ab None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.517219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6429f-37bc-49de-98d8-3b47dc4274ab None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.517533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6429f-37bc-49de-98d8-3b47dc4274ab None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.519608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1244/2457] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:00:49.524701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7197-1a23-4404-b3a6-575bfc1ccb96 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.524968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7197-1a23-4404-b3a6-575bfc1ccb96 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.525256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7197-1a23-4404-b3a6-575bfc1ccb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.544067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4c4a7197-1a23-4404-b3a6-575bfc1ccb96 None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:49.545272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4c4a7197-1a23-4404-b3a6-575bfc1ccb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.551704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1214/2458] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:00:49.563728 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6429f-37bc-49de-98d8-3b47dc4274ab None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:49.575025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6429f-37bc-49de-98d8-3b47dc4274ab None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:49.578517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:49.579152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.579439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.579717 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.588118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:49.589596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30adfcb-9861-4d4e-8e58-599da2e9a7ad service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:49.598797 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1245/2459] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:49.684976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961fa7-0e6e-4cb9-9c3e-8cd9375082fc None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.685349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961fa7-0e6e-4cb9-9c3e-8cd9375082fc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.685703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd961fa7-0e6e-4cb9-9c3e-8cd9375082fc None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.687166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1215/2460] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:00:49.692215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.692514 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.692825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.711597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:49.712844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.722864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1246/2461] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:00:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:00:49.732957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd961fa7-0e6e-4cb9-9c3e-8cd9375082fc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:49.744465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd961fa7-0e6e-4cb9-9c3e-8cd9375082fc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:49.748890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:49.749374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.749657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.749932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.761642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:49.763068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:49.774217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1216/2462] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:49.839462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:49.840880 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.909451 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88694bfe-ca28-4932-9ffe-432a6721566c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:49.914459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:49.915100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:49.915382 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:49.915657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:49.917452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:49.928475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:49.930048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46538837-d585-4a95-be20-872cdd163dbd service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:49.941011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1247/2463] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:51.218734 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:51.228117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d2f6-1f07-46b3-b64a-fce7c5f41c40 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:51.231607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:51.232267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:51.232553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:51.232837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:51.234844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:51.244737 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:51.246200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:51.255043 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1217/2464] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:00:52.017303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:52.017776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:52.017867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:52.035280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:00:52.036539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:52.043571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1248/2465] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:00:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:00:52.055289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:52.067079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df50dc-cf52-4f78-861d-db9842d57aa2 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:52.072673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:52.073305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:52.073592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:52.073872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:52.081740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:52.083936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:52.091731 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1218/2466] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:00:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:00:54.996461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:00:54.998893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:55.015999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050da89e-be70-40a1-9d9c-ad50d64c6c64 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:00:55.026576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:00:55.027248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:00:55.027536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:00:55.027850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:00:55.029772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:00:55.091063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:00:55.092890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f1a0-632c-4bbb-81f3-80b16226fe2b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:00:55.108597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1249/2467] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:01:02.088983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=56 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:01:05.106934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:01:16.418023 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:01:16.432408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d2084-070d-4be3-b8ce-3b3a4ef1d09e service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:01:16.452854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:01:16.453446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:01:16.454244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:01:16.455084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:01:16.457327 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:01:16.476522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:01:16.478283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:01:16.491858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1219/2468] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:01:19.420063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5fb0-546f-4ab4-a1fa-b0651ff5eefc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:01:19.420596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5fb0-546f-4ab4-a1fa-b0651ff5eefc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:01:19.420774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a5fb0-546f-4ab4-a1fa-b0651ff5eefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:01:19.438961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-730a5fb0-546f-4ab4-a1fa-b0651ff5eefc None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:01:19.440204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-730a5fb0-546f-4ab4-a1fa-b0651ff5eefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:01:19.448315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1250/2469] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 14:01:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:01:19.461021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:01:19.472209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b159ef-cfb8-4a12-b416-f1a52f38390c service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:01:19.477189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:01:19.477833 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:01:19.478124 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:01:19.478415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:01:19.486395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:01:19.488005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:01:19.497343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1220/2470] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:01:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:01:29.433814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:01:29.493238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:01:35.754109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:01:35.754637 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:01:35.754826 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:01:35.773320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] MFA Rules not processed for user `532b36f272dc432aad6faa294bd1bc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:01:35.774450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:01:35.781239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1251/2471] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:01:42.105490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:01:42.118305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f22f6e-1e37-4787-8b33-ea8d38b9d1d8 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:01:42.125100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:01:42.125761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:01:42.126048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:01:42.126328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:01:42.128740 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:01:42.141406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:01:42.142995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f879fb-9ef4-4fa1-aa11-035f237d0ead service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:01:42.154088 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1221/2472] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:01:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:01:45.767518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:01:52.149647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:01.951225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:02:01.975756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577ed63-cd3c-4cb2-a707-5805325fa156 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:02:01.983179 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:02:01.984180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:01.984658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:01.985207 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:01.988348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:02.004719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:02:02.006709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:02:02.032090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1252/2473] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:02:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:02:03.665414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:03.665985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:03.665985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:03.683800 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] MFA Rules not processed for user `3acb774848a74e74a5886ee681ed205c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:02:03.685037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:03.740981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1222/2474] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:02:04.055990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:02:04.065459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6064de58-31b4-4c8e-a045-da0f4671da57 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:02:04.069030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:02:04.069675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:04.069963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:04.070242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:04.072061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:04.082194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:02:04.083634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6583a-bb26-4929-bee0-54488b0360e3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:02:04.091807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1253/2475] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:02:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:02:07.486477 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:02:07.488038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:07.504270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf37a6-59ce-4992-bcb4-fdceaa6030cc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:02:07.509648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:02:07.510294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:07.510582 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:07.510861 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:07.512620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:07.523951 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:02:07.525681 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:02:07.535370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1223/2476] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:02:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:02:14.088043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=12 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:17.530422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=31 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:22.430870 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:22.431377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:22.431483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:22.448613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] MFA Rules not processed for user `3eb0459bdac1456a900046e264835a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:02:22.449914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:22.456915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1254/2477] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:02:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:02:22.762807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:02:22.772535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1291d-b872-46f3-811c-62e96d9974b6 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:02:22.776058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:02:22.776698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:22.776986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:22.777270 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:22.779006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:22.788376 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:02:22.789787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c5e64-c9de-412f-bcad-993398046fd4 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:02:22.797598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1224/2478] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:02:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:02:32.443400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:32.794129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:37.616857 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:02:37.630707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9713b-825d-4ad8-aac6-312a75d7b2f1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:02:37.636493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:02:37.637150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:37.637436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:37.637715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:37.639466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:37.697633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:02:37.699235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:02:37.708243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1255/2479] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:02:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:02:45.886920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:02:45.887360 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:02:45.887524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:02:45.906072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] MFA Rules not processed for user `b3560616f4ec496f816602f9c73b69ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:02:45.907272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:02:45.914439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1225/2480] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:02:47.704082 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=28 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:02:55.900725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:07.389766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:07.398945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294902f6-72f0-4b4f-8b45-24baf3423553 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:07.402911 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:07.403581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:07.403897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:07.404181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:07.405945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:07.467743 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:07.469221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:07.479549 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1256/2481] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:10.859575 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:10.863657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:10.878142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef12f-a918-48b4-a662-9d50e375e7e6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:10.883322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:10.884076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:10.884365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:10.884644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:10.886389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:10.900208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:10.902252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:10.914708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1226/2482] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:10.916746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:10.918227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:10.934642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef4a81-7a69-42b9-a8b3-ed5b468a0b9e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:10.939700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:10.940563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:10.940854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:10.941137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] PATH_INFO: `/v3/users/532b36f272dc432aad6faa294bd1bc6c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:10.942351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=532b36f272dc432aad6faa294bd1bc6c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:10.944395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:10.990813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '532b36f272dc432aad6faa294bd1bc6c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:10.997917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '532b36f272dc432aad6faa294bd1bc6c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.010453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '532b36f272dc432aad6faa294bd1bc6c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.011160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '532b36f272dc432aad6faa294bd1bc6c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.011772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e1fb20b4-f688-4362-abe2-3d13b369c8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '532b36f272dc432aad6faa294bd1bc6c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.024542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1257/2483] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:03:10 2022] DELETE /identity/v3/users/532b36f272dc432aad6faa294bd1bc6c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:03:11.194208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.205277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8c6bc-ce1c-43b9-b527-4d636b02489e service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.262411 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.263276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.263581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.263867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] PATH_INFO: `/v3/projects/54ee7d13408645e6b6517f541fe45670` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.265028 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ee7d13408645e6b6517f541fe45670)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.267247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.314304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ee7d13408645e6b6517f541fe45670', 'request_id': 'req-b17f4ed5-0e06-4630-80d4-9565efc13832'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.320454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ee7d13408645e6b6517f541fe45670', 'request_id': 'req-b17f4ed5-0e06-4630-80d4-9565efc13832'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.325898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ee7d13408645e6b6517f541fe45670', 'request_id': 'req-b17f4ed5-0e06-4630-80d4-9565efc13832'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.337557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] The token cache is being invalidate because project 54ee7d13408645e6b6517f541fe45670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:03:11.337852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.338489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b17f4ed5-0e06-4630-80d4-9565efc13832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:11.340073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1227/2484] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:03:11 2022] DELETE /identity/v3/projects/54ee7d13408645e6b6517f541fe45670 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:11.346891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.347189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.347479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.365998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:11.367199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:11.425018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1258/2485] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:11.433097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.433441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.433730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.451805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:11.453433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:11.462824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1228/2486] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:11.467023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.477473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04283b7-5fba-417b-96ea-9dc9216d0928 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.481954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.482600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.482887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.483163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.483939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:03:11.484247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.486248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.487710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:11.503697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1259/2487] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:03:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:03:11.504839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.514169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490f1f5-3a62-44e3-86f6-4c46115e9727 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.519418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.520112 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.520403 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.520686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.521391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.523323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.557677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1229/2488] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:03:11 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:03:11.561848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.572245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4c144-e4d5-4188-88ed-9c685c0b5cad admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.629903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.630576 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.630868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.631182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.631901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.633843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.653322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1260/2489] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:03:11 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:11.657617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.667735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9538b1-3b73-44e8-97fb-059250075b25 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.672350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.673591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.673878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.674154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.674885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.676514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.686576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1230/2490] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:11.691224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.701269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea340ce-5e18-4423-a59c-3b9034cfaae9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.706113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.706714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.707007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.707292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.708069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.709674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd130d-a825-420a-8f7d-4680cdac4850 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.719433 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1261/2491] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:11.724744 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.736262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178d80c-5eaa-404a-bcbb-b9f38e941862 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.740960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.741851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.742139 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.742690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] PATH_INFO: `/v3/projects/afb964eae6b941fc943cca8cf9c3d075/users/5f29e83de728455bac6a1227e1f2deef/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.758209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb964eae6b941fc943cca8cf9c3d075, user_id=5f29e83de728455bac6a1227e1f2deef, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.760890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.774524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1231/2492] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:03:11 2022] PUT /identity/v3/projects/afb964eae6b941fc943cca8cf9c3d075/users/5f29e83de728455bac6a1227e1f2deef/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:11.785339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.785636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.785919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.812064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] MFA Rules not processed for user `5f29e83de728455bac6a1227e1f2deef`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:11.813264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:11.876906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1262/2493] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:11.887786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:11.889519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:11.896370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:11.968486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141e5eb-b31a-45c3-886b-ff42a24f7883 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:11.974755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:11.975415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:11.975704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:11.976014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:11.983900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:11.986090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ceb59-5fd3-4f54-8d59-7c18ac3db5a1 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:11.995383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1232/2494] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:14.646251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:14.706323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da1e4c-df06-46a0-8e67-0e8a16a968dd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:14.710725 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:14.711575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:14.711882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:14.712162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] PATH_INFO: `/v3/users/5f29e83de728455bac6a1227e1f2deef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:14.713332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f29e83de728455bac6a1227e1f2deef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:14.715243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:14.717569 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:14.771137 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f29e83de728455bac6a1227e1f2deef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:14.777782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f29e83de728455bac6a1227e1f2deef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:14.789368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f29e83de728455bac6a1227e1f2deef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:14.790034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f29e83de728455bac6a1227e1f2deef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:14.790636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f29e83de728455bac6a1227e1f2deef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:14.803452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1263/2495] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:03:14 2022] DELETE /identity/v3/users/5f29e83de728455bac6a1227e1f2deef => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:14.808144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:14.808605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:14.808786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:14.826021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:14.827245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:14.884755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1233/2496] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:14.895196 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:14.896614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:14.952793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4fd8a-da02-4cdc-beee-a4299f826b6c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:14.957009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:14.957651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:14.957941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:14.958221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:14.967200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:14.968687 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d11de0a-b315-419f-bf51-102bc35c0b53 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:14.978152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1264/2497] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:15.255123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.256559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:15.269281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ca674-6a40-4747-bfb6-574cf343168f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.274206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.275104 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.275398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.275679 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] PATH_INFO: `/v3/projects/afb964eae6b941fc943cca8cf9c3d075` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.276914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb964eae6b941fc943cca8cf9c3d075)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.278840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.326625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb964eae6b941fc943cca8cf9c3d075', 'request_id': 'req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:15.334242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb964eae6b941fc943cca8cf9c3d075', 'request_id': 'req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:15.339922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb964eae6b941fc943cca8cf9c3d075', 'request_id': 'req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:15.355111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] The token cache is being invalidate because project afb964eae6b941fc943cca8cf9c3d075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:03:15.355412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:15.356115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5d4e9e09-ca71-4f28-b96d-10e2a78871e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:15.357656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1234/2498] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:03:15 2022] DELETE /identity/v3/projects/afb964eae6b941fc943cca8cf9c3d075 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:15.362859 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.363156 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.363434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.381443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:15.382652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:15.441298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1265/2499] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:15.449997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.450293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.450579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.470509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:15.471695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:15.479614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1235/2500] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:15.483733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.493934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c48f3-7677-469d-885b-7e61382c77ba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.498589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.499249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.499537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.499849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.501125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:03:15.501599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.503221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.515039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1266/2501] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:03:15.519916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.529074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203ca2-abfe-4370-a89a-9a5d9f957978 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.534452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.535110 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.535394 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.535672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.536413 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.538293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.541498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:15.566050 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1236/2502] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:03:15 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:15.570098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.580170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fedce7-2954-493b-8c19-40fab4d0f3aa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.587887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:15.638651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.639322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.639945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.640228 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.640915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.642823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.666046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1267/2503] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:03:15 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:15.670281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.681165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bc5b8-3a3c-432a-91e6-86de97e5cfaf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.685758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.686416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.686703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.686983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.687714 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.689584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.699316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1237/2504] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:15.703284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.713243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1521452-ebda-4fd7-a29f-333dc883e67e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.717806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.718460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.718748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.719030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.720304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.721972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe3577-69f9-46ae-819d-81f1c96582f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.732025 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1268/2505] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:15.737368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.750305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cc298-cc7e-4832-a79d-a97bec5e1f64 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.755296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.756215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.756504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.756786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] PATH_INFO: `/v3/projects/1a3abbecd01143bb80f39c1697d965b8/users/3222dfb8ceb94ae3847101ba124f9464/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.771349 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a3abbecd01143bb80f39c1697d965b8, user_id=3222dfb8ceb94ae3847101ba124f9464, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.774030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.787757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1238/2506] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:03:15 2022] PUT /identity/v3/projects/1a3abbecd01143bb80f39c1697d965b8/users/3222dfb8ceb94ae3847101ba124f9464/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:03:15.798721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.799020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.799304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.826060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] MFA Rules not processed for user `3222dfb8ceb94ae3847101ba124f9464`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:15.827187 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:15.889939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1269/2507] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:15.901195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:15.902994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:15.959230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa6067-b8ca-4b9b-a4b0-bf488facb837 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:15.962874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:15.963517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:15.963805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:15.964117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:15.973195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:15.974623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03646134-e4e2-4f96-ac77-bdb3fe0ee561 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:15.983318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1239/2508] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:24.282759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:24.343507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ab325-87f4-43b3-8a0a-87382934116f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:24.348335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:24.349201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:24.349492 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:24.349772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] PATH_INFO: `/v3/users/3222dfb8ceb94ae3847101ba124f9464` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:24.350958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3222dfb8ceb94ae3847101ba124f9464)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:24.352938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:24.374742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:24.401252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3222dfb8ceb94ae3847101ba124f9464'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.408043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3222dfb8ceb94ae3847101ba124f9464'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.420649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3222dfb8ceb94ae3847101ba124f9464'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.421345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3222dfb8ceb94ae3847101ba124f9464'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.421948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3222dfb8ceb94ae3847101ba124f9464'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.434338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1270/2509] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:03:24 2022] DELETE /identity/v3/users/3222dfb8ceb94ae3847101ba124f9464 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:24.438810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:24.439285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:24.439385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:24.457096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:24.458436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:24.516995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1240/2510] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:24.527853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:24.529276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:24.585533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658effa-7931-485b-9d55-a100faaa2605 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:24.589167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:24.589813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:24.590104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:24.590386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:24.599521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:24.601007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb91c1a-c913-4a5c-9e62-3fae3a9995ae service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:24.610220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1271/2511] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:24.887304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:24.888834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:24.901042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3c863-4043-4b15-974e-27c589f64c4b None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:24.908203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:24.909114 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:24.909416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:24.909675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] PATH_INFO: `/v3/projects/1a3abbecd01143bb80f39c1697d965b8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:24.911146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a3abbecd01143bb80f39c1697d965b8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:24.913669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:24.957912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:24.959126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a3abbecd01143bb80f39c1697d965b8', 'request_id': 'req-429bded3-eb2b-4944-ac9c-01a5ab5bacde'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.966278 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a3abbecd01143bb80f39c1697d965b8', 'request_id': 'req-429bded3-eb2b-4944-ac9c-01a5ab5bacde'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.971925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a3abbecd01143bb80f39c1697d965b8', 'request_id': 'req-429bded3-eb2b-4944-ac9c-01a5ab5bacde'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.983323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] The token cache is being invalidate because project 1a3abbecd01143bb80f39c1697d965b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:03:24.983611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.984259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-429bded3-eb2b-4944-ac9c-01a5ab5bacde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:03:24.985772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1241/2512] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:03:24 2022] DELETE /identity/v3/projects/1a3abbecd01143bb80f39c1697d965b8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:03:24.991245 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:24.991546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:24.992093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.009738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:25.010920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:25.068244 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1272/2513] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:25.076314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.076728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.077015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.095134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:25.096352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:25.104373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1242/2514] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:25.108356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.118861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a1fc1-90da-4de0-b8ea-06359dec9b2e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.123770 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.124474 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.124765 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.125049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.125811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:03:25.126118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.127721 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.141651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1273/2515] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:03:25.145606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.155657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa49c6-f423-4332-bb7a-472eb7e24ac7 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.161382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.162039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.162325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.162607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.163299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.165227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.190973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1243/2516] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:25.195256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.205167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a49d5-7964-4640-b3f3-8a4046b3e5bc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.231406 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:25.264344 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.265014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.265305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.265586 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.266283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.268226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.288186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1274/2517] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:25.292223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.303221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a27596-ea86-473d-bbd1-ebbc49e93927 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.307795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.308476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.308773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.309057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.309818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.311412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.321931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1244/2518] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:25.325597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.335836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cae8ad-3949-453d-b7cb-cd442da4eab3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.340709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.341373 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.341664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.341944 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.342752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.345268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487e684-f806-455d-adbb-4a3935e8f353 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.358498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1275/2519] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:25.362479 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.373919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85a595-7e86-4274-bc77-ab9446dcf40a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.378534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.379416 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.379707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.380036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] PATH_INFO: `/v3/projects/f6fb10ff58e94ac6b27457c4396ab780/users/7e3e4041c514433bbab5514cb3a428a7/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.395523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fb10ff58e94ac6b27457c4396ab780, user_id=7e3e4041c514433bbab5514cb3a428a7, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.398211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.411265 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1245/2520] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:03:25 2022] PUT /identity/v3/projects/f6fb10ff58e94ac6b27457c4396ab780/users/7e3e4041c514433bbab5514cb3a428a7/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:25.423630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.423952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.424235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.450304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] MFA Rules not processed for user `7e3e4041c514433bbab5514cb3a428a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:25.451503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:25.514467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1276/2521] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:25.519587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.579795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6411355-d3ff-462f-8728-ebcd14c7d30d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.584894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.585543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.585830 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.586106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.586795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.588712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.613233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1246/2522] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:03:25.617475 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.628032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42a21d-fdf8-470f-a943-fdb23f9ed6de None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.685539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.686211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.686502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.686783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.687479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.689437 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.708453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1277/2523] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:03:25.712348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.723036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3bde9-a54b-43d2-a2e4-cf20f90e7f1e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.727657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.728346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.728636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.728917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.729648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.731237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.742294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1247/2524] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:25.746024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.755850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29da15-5b8c-4319-969f-d2492aff4d4a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.761121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.762102 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.762477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.762844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] PATH_INFO: `/v3/projects/175247db407847ff8667a0115ec761bf/users/11e17fd12467444187ab5be89eafde19/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.779157 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175247db407847ff8667a0115ec761bf, user_id=11e17fd12467444187ab5be89eafde19, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.781830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.795370 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1278/2525] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:03:25 2022] PUT /identity/v3/projects/175247db407847ff8667a0115ec761bf/users/11e17fd12467444187ab5be89eafde19/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:25.799048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.809009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46f4eb-b315-4d7b-ad9a-8c4681a1cf00 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.853128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:25.868208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.868879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.869171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.869455 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.870188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.871852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f12b3-dc1b-47c2-9e26-9c082e010915 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.881864 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1248/2526] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:03:25.886027 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:25.891361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:25.898686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94346393-da8e-4d17-9241-4b84970cff45 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:25.903472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:25.904397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.904690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.905003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] PATH_INFO: `/v3/projects/175247db407847ff8667a0115ec761bf/users/11e17fd12467444187ab5be89eafde19/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.906984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=175247db407847ff8667a0115ec761bf, user_id=11e17fd12467444187ab5be89eafde19, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:25.910461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:25.921298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1279/2527] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:03:25 2022] PUT /identity/v3/projects/175247db407847ff8667a0115ec761bf/users/11e17fd12467444187ab5be89eafde19/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:03:25.931891 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:25.932190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:25.932478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:25.960099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] MFA Rules not processed for user `11e17fd12467444187ab5be89eafde19`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:03:25.961308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:26.027456 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1249/2528] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:03:26.038546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:26.040295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:26.096883 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c357c2-1044-45bb-90d9-dbb5819d4bd5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:26.101279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:26.101921 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:26.102208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:26.102488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:26.162843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:26.165145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:26.175592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1280/2529] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:32.849385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:32.858900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9b7c-b675-4676-b9d3-0aaacb586280 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:32.862635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:32.864070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:32.864362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:32.864648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:32.873958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:32.875373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:32.884815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1250/2530] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:36.171008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:38.512377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:03:38.517479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:38.585441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063cb0ce-d24a-4ab3-99a8-da9b31e55694 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:03:38.590328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:03:38.591026 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:03:38.591314 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:03:38.591622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:03:38.593509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:03:38.605249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:03:38.606888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693797-08c3-47e3-a6aa-60566e1800e2 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:03:38.617525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1281/2531] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:03:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:03:42.880601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=48 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:03:48.613004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=32 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:01.726444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:01.789968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04985a9-45a7-4590-b355-87d9bf5d0ecd service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:01.794331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:01.795203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:01.795493 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:01.795774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] PATH_INFO: `/v3/users/7e3e4041c514433bbab5514cb3a428a7` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:01.796991 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3e4041c514433bbab5514cb3a428a7)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:01.798903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:01.844693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3e4041c514433bbab5514cb3a428a7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:01.845339 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3e4041c514433bbab5514cb3a428a7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:01.857274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3e4041c514433bbab5514cb3a428a7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:01.857896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3e4041c514433bbab5514cb3a428a7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:01.865053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3e4041c514433bbab5514cb3a428a7'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:01.877904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1251/2532] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:01 2022] DELETE /identity/v3/users/7e3e4041c514433bbab5514cb3a428a7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:01.883125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:01.883622 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:01.883729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:01.900975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:01.902167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:01.961506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1282/2533] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:01.971332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:01.972878 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:02.029288 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff256f26-7132-47b6-a00e-e0d767fecd9c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:02.032922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:02.033559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:02.033843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:02.034118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:02.043136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:02.044519 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:02.053866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1252/2534] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:02.316993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:02.319168 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:02.331533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7976d77f-d287-4c5e-95a3-36048cee0365 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:02.336254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:02.337126 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:02.337416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:02.337704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] PATH_INFO: `/v3/projects/f6fb10ff58e94ac6b27457c4396ab780` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:02.338964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fb10ff58e94ac6b27457c4396ab780)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:02.341010 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:02.387432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fb10ff58e94ac6b27457c4396ab780', 'request_id': 'req-8f25d7d9-7039-48c2-954a-9daf1bc674f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.393592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fb10ff58e94ac6b27457c4396ab780', 'request_id': 'req-8f25d7d9-7039-48c2-954a-9daf1bc674f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.399311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fb10ff58e94ac6b27457c4396ab780', 'request_id': 'req-8f25d7d9-7039-48c2-954a-9daf1bc674f9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.411498 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] The token cache is being invalidate because project f6fb10ff58e94ac6b27457c4396ab780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:02.411781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.412423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.413949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1283/2535] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:02 2022] DELETE /identity/v3/projects/f6fb10ff58e94ac6b27457c4396ab780 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:02.417957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:02.428063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a050187-a84a-49ce-8c41-db54d24e4df6 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:02.446384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:02.485314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:02.486185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:02.486474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:02.486755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] PATH_INFO: `/v3/users/11e17fd12467444187ab5be89eafde19` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:02.487977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e17fd12467444187ab5be89eafde19)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:02.490440 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:02.537441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e17fd12467444187ab5be89eafde19'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.538184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e17fd12467444187ab5be89eafde19'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.549787 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e17fd12467444187ab5be89eafde19'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.550448 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e17fd12467444187ab5be89eafde19'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.557761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-af10f53e-3bef-45fc-944e-a2306176ff40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e17fd12467444187ab5be89eafde19'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.569586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1253/2536] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:02 2022] DELETE /identity/v3/users/11e17fd12467444187ab5be89eafde19 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:02.832827 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:02.834281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:02.847217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25d7d9-7039-48c2-954a-9daf1bc674f9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:02.905030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:02.905898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:02.906186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:02.906465 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] PATH_INFO: `/v3/projects/175247db407847ff8667a0115ec761bf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:02.907700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175247db407847ff8667a0115ec761bf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:02.909640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:02.939900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:02.959040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175247db407847ff8667a0115ec761bf', 'request_id': 'req-6454cb93-e57a-4bb2-b315-fb034c8c3f60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.965073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175247db407847ff8667a0115ec761bf', 'request_id': 'req-6454cb93-e57a-4bb2-b315-fb034c8c3f60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.970479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175247db407847ff8667a0115ec761bf', 'request_id': 'req-6454cb93-e57a-4bb2-b315-fb034c8c3f60'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.984697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] The token cache is being invalidate because project 175247db407847ff8667a0115ec761bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:02.984989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.985615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6454cb93-e57a-4bb2-b315-fb034c8c3f60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:02.987164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1284/2537] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:02 2022] DELETE /identity/v3/projects/175247db407847ff8667a0115ec761bf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:02.992311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:02.992761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:02.992875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.010643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:03.011886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:03.070793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1254/2538] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:03.079663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.079951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.080260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.100479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:03.101652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:03.109631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1285/2539] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:03.113854 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.123720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433c224-f097-4b66-892a-b8aadbdedbd4 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.128234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.128884 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.129173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.129636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.130379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:04:03.130687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.132300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.143704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1255/2540] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:04:03.148529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.157606 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c887b13f-807f-46f6-86cb-f494f5c05129 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.163017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.163658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.163978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.164282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.164980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.166860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.191329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1286/2541] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/projects => generated 381 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:03.195365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.205140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a6bb6-1a0f-4760-a20c-772ff9b58602 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.231283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:03.263471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.264196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.264486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.264771 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.265471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.267355 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.287309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1256/2542] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:03.291363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.301412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f94f-0d83-4b04-a729-7852fcde6d4d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.306264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.306918 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.307202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.307486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.308271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.309867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.320799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1287/2543] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:03.324464 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.335017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1289b6-4896-4904-a5c3-41768cbe9f47 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.339523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.340212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.340505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.340786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.341508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.343096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d0d4e-4bf4-4a9f-9469-4746fe5abb9d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.354887 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1257/2544] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:03.358936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.369211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1224f9b-3cbd-49df-9dc1-61e0a8cffcf5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.373832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.374705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.374994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.375277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] PATH_INFO: `/v3/projects/c018a6bc78b14cf082e7dc9b114fd973/users/934d8dc39fb149fcb9edbc0391c9388f/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.390383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c018a6bc78b14cf082e7dc9b114fd973, user_id=934d8dc39fb149fcb9edbc0391c9388f, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.393096 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.407181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1288/2545] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:03 2022] PUT /identity/v3/projects/c018a6bc78b14cf082e7dc9b114fd973/users/934d8dc39fb149fcb9edbc0391c9388f/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:03.417498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.417798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.418087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.444669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] MFA Rules not processed for user `934d8dc39fb149fcb9edbc0391c9388f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:03.445917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:03.509322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1258/2546] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:03.513814 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.573626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff825aa-d240-4a3c-af13-562fc4196765 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.578270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.578950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.579241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.579525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.580269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.582217 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.607596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1289/2547] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:03.611554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.623409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9afe1-83e8-485d-925a-38003043a39f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.681247 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.681933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.682232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.682514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.683220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.685191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.705993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1259/2548] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:03.712092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.722738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a52c8-fea2-4d97-90cd-a6204f39938a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.727663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.728353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.728641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.728920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.729642 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.731239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.742705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1290/2549] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:03.746665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.757383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6115de-f203-4496-9fec-d5f585ff0612 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.762325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.763233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.763517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.763795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] PATH_INFO: `/v3/projects/ee444b8aa0f742ca94bcba381ac94d32/users/579140f05db74972bf48275380bfa894/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.778755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee444b8aa0f742ca94bcba381ac94d32, user_id=579140f05db74972bf48275380bfa894, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.781400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.794696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1260/2550] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:03 2022] PUT /identity/v3/projects/ee444b8aa0f742ca94bcba381ac94d32/users/579140f05db74972bf48275380bfa894/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:03.798703 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.809311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca4117-d99c-4bcf-8f78-75213d6b5170 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.829575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:03.871534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.872253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.872644 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.872956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.873692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.875299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d62ddb0-987c-404f-a19b-c136bdc1f431 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.886122 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1291/2551] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:03.890383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:03.895931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:03.902510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853820f-866f-4731-b1d4-f18d4674772f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:03.908390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:03.909517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.909813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.910098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] PATH_INFO: `/v3/projects/ee444b8aa0f742ca94bcba381ac94d32/users/579140f05db74972bf48275380bfa894/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.912217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee444b8aa0f742ca94bcba381ac94d32, user_id=579140f05db74972bf48275380bfa894, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:03.914977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:03.925437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1261/2552] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:03 2022] PUT /identity/v3/projects/ee444b8aa0f742ca94bcba381ac94d32/users/579140f05db74972bf48275380bfa894/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:03.937007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:03.937446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:03.937544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:03.963838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] MFA Rules not processed for user `579140f05db74972bf48275380bfa894`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:03.965052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:04.033872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1292/2553] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:04.046698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.048435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:04.105014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ebcf6-9e4f-4039-8312-36bc86a2bdff admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:04.108632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:04.109273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:04.109568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:04.109851 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:04.119606 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:04.121024 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:04.130473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1262/2554] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:04.426774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.435933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d460a-af37-45ae-8305-14597c7335f5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:04.440459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '579140f05db74972bf48275380bfa894', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee444b8aa0f742ca94bcba381ac94d32', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:04.441113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:04.441400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:04.441683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:04.442760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:04:04.443075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:04.444706 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:04.455854 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1293/2555] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:04:04.460757 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.469657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43443222-c61c-404f-b397-1bf637140b83 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:04.475267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '579140f05db74972bf48275380bfa894', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee444b8aa0f742ca94bcba381ac94d32', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:04.475964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:04.476261 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:04.476550 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:04.477259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:04.479165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:04.505461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1263/2556] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:04 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:04:04.509591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.519707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22921-4a76-4324-9117-428cbe56b807 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:04.574822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '579140f05db74972bf48275380bfa894', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee444b8aa0f742ca94bcba381ac94d32', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:04.575476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:04.575768 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:04.576083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:04.576799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:04:04.577115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:04.578709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:04.590318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1294/2557] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 14:04:04.627058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.637135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142d58e-7880-4552-b3a3-b36c0c4cd9a6 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:04.641706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '579140f05db74972bf48275380bfa894', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee444b8aa0f742ca94bcba381ac94d32', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:04.642509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:04.642809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:04.643091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] PATH_INFO: `/v3/projects/c3e0d38b1a8645dba026e219c407acac` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:04.644296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c3e0d38b1a8645dba026e219c407acac)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:04.646711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:04.691496 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e0d38b1a8645dba026e219c407acac', 'request_id': 'req-161ace18-6967-4aec-9d4e-3137db78c2e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:04.697675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e0d38b1a8645dba026e219c407acac', 'request_id': 'req-161ace18-6967-4aec-9d4e-3137db78c2e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:04.703230 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e0d38b1a8645dba026e219c407acac', 'request_id': 'req-161ace18-6967-4aec-9d4e-3137db78c2e4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:04.715898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] The token cache is being invalidate because project c3e0d38b1a8645dba026e219c407acac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:04.716188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:04.716817 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-161ace18-6967-4aec-9d4e-3137db78c2e4 tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:04.718337 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1264/2558] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:04 2022] DELETE /identity/v3/projects/c3e0d38b1a8645dba026e219c407acac => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:04.903198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:04.904934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:04.936018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:04.968977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5df36-ca79-4629-9ebe-00af27b7f14e tempest-VolumeQuotaClassesTest-656654255 tempest-VolumeQuotaClassesTest-656654255-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:05.024321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:05.025248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:05.025542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:05.025823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] PATH_INFO: `/v3/users/934d8dc39fb149fcb9edbc0391c9388f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:05.027019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] RBAC: Authorizing `identity:delete_user(user_id=934d8dc39fb149fcb9edbc0391c9388f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:05.028982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:05.074722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934d8dc39fb149fcb9edbc0391c9388f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.082103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934d8dc39fb149fcb9edbc0391c9388f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.093472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934d8dc39fb149fcb9edbc0391c9388f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.094153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934d8dc39fb149fcb9edbc0391c9388f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.094755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934d8dc39fb149fcb9edbc0391c9388f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.108969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1295/2559] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:04 2022] DELETE /identity/v3/users/934d8dc39fb149fcb9edbc0391c9388f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:05.113612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:05.114743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:05.115036 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:05.135600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:05.136855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:05.183025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:05.196704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1265/2560] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:05.206237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:05.208232 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:05.265394 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f19960-7995-4d1d-bbbb-673cc31a562f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:05.269202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:05.269848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:05.270978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:05.271517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:05.278633 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:05.280962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:05.282970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:05.298697 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1296/2561] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:05.578913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:05.580369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:05.593414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad99bb-3a9a-4ea3-873b-97deb470724d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:05.598072 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:05.599184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:05.599476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:05.599762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] PATH_INFO: `/v3/projects/c018a6bc78b14cf082e7dc9b114fd973` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:05.600998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c018a6bc78b14cf082e7dc9b114fd973)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:05.602904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:05.650159 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c018a6bc78b14cf082e7dc9b114fd973', 'request_id': 'req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.656779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c018a6bc78b14cf082e7dc9b114fd973', 'request_id': 'req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.665037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c018a6bc78b14cf082e7dc9b114fd973', 'request_id': 'req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.681488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] The token cache is being invalidate because project c018a6bc78b14cf082e7dc9b114fd973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:05.682011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.682774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.684396 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1266/2562] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:05 2022] DELETE /identity/v3/projects/c018a6bc78b14cf082e7dc9b114fd973 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:05.688499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:05.698664 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3927e0df-6022-43fa-b0e1-10d90e8a10c1 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:05.753938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:05.755367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:05.755754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:05.756068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] PATH_INFO: `/v3/users/579140f05db74972bf48275380bfa894` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:05.757250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=579140f05db74972bf48275380bfa894)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:05.759180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:05.804005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579140f05db74972bf48275380bfa894'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.810689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579140f05db74972bf48275380bfa894'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.822772 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579140f05db74972bf48275380bfa894'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.823452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579140f05db74972bf48275380bfa894'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.824097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c119ff46-ecfe-4423-bbf8-5c0ee6d1e78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579140f05db74972bf48275380bfa894'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:05.837053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1297/2563] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:05 2022] DELETE /identity/v3/users/579140f05db74972bf48275380bfa894 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:04:06.109175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.111449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:06.126249 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7b13e-c5b5-4617-96a1-6c754f81b98e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.188086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.188949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.189240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.189520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] PATH_INFO: `/v3/projects/ee444b8aa0f742ca94bcba381ac94d32` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.190689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee444b8aa0f742ca94bcba381ac94d32)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.193253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.236964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee444b8aa0f742ca94bcba381ac94d32', 'request_id': 'req-e08f3c38-52be-4fbf-8de5-95799ef522a2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:06.243993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee444b8aa0f742ca94bcba381ac94d32', 'request_id': 'req-e08f3c38-52be-4fbf-8de5-95799ef522a2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:06.249770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee444b8aa0f742ca94bcba381ac94d32', 'request_id': 'req-e08f3c38-52be-4fbf-8de5-95799ef522a2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:06.262621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] The token cache is being invalidate because project ee444b8aa0f742ca94bcba381ac94d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:06.262916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:06.263545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e08f3c38-52be-4fbf-8de5-95799ef522a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:06.265091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1267/2564] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:06 2022] DELETE /identity/v3/projects/ee444b8aa0f742ca94bcba381ac94d32 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:06.270914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.271426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.271536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.290047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:06.291248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:06.356178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1298/2565] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:06.364502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.364798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.365173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.377733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:06.384415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:06.386206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:06.394040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1268/2566] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:06.398711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.408264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b3daa-7adc-4f77-a4e4-159cb662d68a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.413410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.414060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.414349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.414628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.415350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:04:06.415661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.417280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.429149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1299/2567] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:04:06.433310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.442641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a3238-bd16-41ef-8809-526f0a94533e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.447318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.447999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.448290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.448574 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.449272 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.451180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.477317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1269/2568] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:06.481483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.486499 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:06.493412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c372a7-1153-41d4-888c-afddcd70e4b0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.550147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.550821 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.551109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.551522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.552213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.554128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.573801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1300/2569] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:06.577900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.588693 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368562a-cc70-4f3f-b4e5-4fb1529a2eaf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.593326 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.593989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.594284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.594571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.595305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.596980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.607310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1270/2570] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:06.611115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.621380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852a476-bdf7-4164-a35b-d027a9a3f870 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.625958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.626611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.626900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.627181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.627947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.630094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d93715-0ce3-43ac-9f12-ca923d7f508d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.640337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1301/2571] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:06.644285 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.654860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e750-d0bd-4828-afb8-3c78259d9132 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.659636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.660783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.661300 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.661618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] PATH_INFO: `/v3/projects/13a4efb22fcd4e6898b9936ce193e75b/users/853cc1749a8844b6ab36b404c032bad6/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.675912 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a4efb22fcd4e6898b9936ce193e75b, user_id=853cc1749a8844b6ab36b404c032bad6, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.679097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.693224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1271/2572] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:06 2022] PUT /identity/v3/projects/13a4efb22fcd4e6898b9936ce193e75b/users/853cc1749a8844b6ab36b404c032bad6/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:06.703432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.703748 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.704064 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.730139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] MFA Rules not processed for user `853cc1749a8844b6ab36b404c032bad6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:06.731375 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:06.794761 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1302/2573] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:06.799141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.860005 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a68c56-5f42-4d4e-96f1-ae95f310f701 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.865219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.866120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.866406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.866687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.867405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.869331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.894491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1272/2574] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:06.898714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:06.909190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327efaf-d7dc-4881-b604-c8853d6d31f9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:06.964919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:06.965674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:06.965966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:06.966255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:06.966963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:06.968936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:06.984161 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:06.993760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1303/2575] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:06 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:06.994717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.005706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05329490-0894-4004-8630-4e5c30d93c66 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.010335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.010998 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.011299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.011587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.012363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.013960 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.024395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1273/2576] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:07.028462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.038736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ade60-a220-443b-8783-4a1653415f7c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.043335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.044345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.044641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.044971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] PATH_INFO: `/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d/users/8e7dbdbcabb94f42b8ba9bf59189fd40/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.060112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0b2148b9246fe999d36f3f41d5c8d, user_id=8e7dbdbcabb94f42b8ba9bf59189fd40, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.062808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.075623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1304/2577] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:07 2022] PUT /identity/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d/users/8e7dbdbcabb94f42b8ba9bf59189fd40/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:07.079467 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.089764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919d50c-aaa5-49a2-9a91-085dcbbaf727 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.145565 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.146336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.146632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.146917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.147698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.149314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61908f2f-b710-4e91-b015-10edd42f596e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.159009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1274/2578] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:07.163919 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.174097 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674cf63-7955-4097-ae5d-8e125afd70d3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.179846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.180719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.181008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.181287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] PATH_INFO: `/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d/users/8e7dbdbcabb94f42b8ba9bf59189fd40/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.183227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0b2148b9246fe999d36f3f41d5c8d, user_id=8e7dbdbcabb94f42b8ba9bf59189fd40, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.185926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.196141 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1305/2579] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:07 2022] PUT /identity/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d/users/8e7dbdbcabb94f42b8ba9bf59189fd40/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:07.207073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.207383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.207673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.231628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:07.234665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] MFA Rules not processed for user `8e7dbdbcabb94f42b8ba9bf59189fd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:07.236504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:07.305869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1275/2580] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:07.316470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.318427 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:07.377160 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd915e-b8ca-4bb4-8a30-ec09e1fa25e1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.380941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.381615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.381892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.382175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.391234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.392758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b31033-55a5-4368-9a61-dd50ecf59925 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.402782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1306/2581] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:07.785503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:07.846804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b88fd-2232-45cb-8121-42fafcafe71e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:07.851552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:07.852451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.852738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.853013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] PATH_INFO: `/v3/users/853cc1749a8844b6ab36b404c032bad6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.854177 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853cc1749a8844b6ab36b404c032bad6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:07.856096 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:07.902533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853cc1749a8844b6ab36b404c032bad6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:07.903187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853cc1749a8844b6ab36b404c032bad6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:07.914966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853cc1749a8844b6ab36b404c032bad6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:07.915597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853cc1749a8844b6ab36b404c032bad6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:07.922311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853cc1749a8844b6ab36b404c032bad6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:07.936209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1276/2582] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:07 2022] DELETE /identity/v3/users/853cc1749a8844b6ab36b404c032bad6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:07.941113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:07.941430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:07.941711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:07.960350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:07.961536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:08.019229 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1307/2583] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:08.029837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:08.031232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:08.051042 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:08.091419 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a0096-f388-4c1a-849c-558d9c3a9dd4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:08.095154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:08.095803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:08.096127 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:08.096412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:08.105510 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:08.106888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:08.116498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1277/2584] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:08.381702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:08.383231 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:08.395988 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169fa17-48ec-4d1b-a3b6-c3cae8ab1a9a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:08.400684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:08.401547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:08.402004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:08.402287 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] PATH_INFO: `/v3/projects/13a4efb22fcd4e6898b9936ce193e75b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:08.403481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a4efb22fcd4e6898b9936ce193e75b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:08.405438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:08.451440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a4efb22fcd4e6898b9936ce193e75b', 'request_id': 'req-63c61c29-ccd8-4f72-ae05-23aa24d24b83'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.458225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a4efb22fcd4e6898b9936ce193e75b', 'request_id': 'req-63c61c29-ccd8-4f72-ae05-23aa24d24b83'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.462938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:08.464265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a4efb22fcd4e6898b9936ce193e75b', 'request_id': 'req-63c61c29-ccd8-4f72-ae05-23aa24d24b83'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.476698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] The token cache is being invalidate because project 13a4efb22fcd4e6898b9936ce193e75b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:08.476991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.477617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.479145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1308/2585] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:08 2022] DELETE /identity/v3/projects/13a4efb22fcd4e6898b9936ce193e75b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:08.483223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:08.493572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7c12c-5c1b-4345-bddc-83a161494867 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:08.549544 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:08.550431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:08.550723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:08.551008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] PATH_INFO: `/v3/users/8e7dbdbcabb94f42b8ba9bf59189fd40` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:08.552927 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7dbdbcabb94f42b8ba9bf59189fd40)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:08.554850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:08.600256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7dbdbcabb94f42b8ba9bf59189fd40'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.600903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7dbdbcabb94f42b8ba9bf59189fd40'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.612457 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7dbdbcabb94f42b8ba9bf59189fd40'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.613080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7dbdbcabb94f42b8ba9bf59189fd40'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.620711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-3226b323-4155-434c-a5e5-014661b0cfb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7dbdbcabb94f42b8ba9bf59189fd40'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:08.633377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1278/2586] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:04:08 2022] DELETE /identity/v3/users/8e7dbdbcabb94f42b8ba9bf59189fd40 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:08.893704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:08.895873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:08.908929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c61c29-ccd8-4f72-ae05-23aa24d24b83 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:08.965790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:08.966652 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:08.966941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:08.967220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] PATH_INFO: `/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:08.968436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a0b2148b9246fe999d36f3f41d5c8d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:08.970348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.014952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a0b2148b9246fe999d36f3f41d5c8d', 'request_id': 'req-e0dd692a-c03c-4085-9d29-edf965478129'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:09.021219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a0b2148b9246fe999d36f3f41d5c8d', 'request_id': 'req-e0dd692a-c03c-4085-9d29-edf965478129'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:09.026628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a0b2148b9246fe999d36f3f41d5c8d', 'request_id': 'req-e0dd692a-c03c-4085-9d29-edf965478129'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:09.038301 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] The token cache is being invalidate because project b9a0b2148b9246fe999d36f3f41d5c8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:04:09.038591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:09.039211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e0dd692a-c03c-4085-9d29-edf965478129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:04:09.040727 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1309/2587] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:04:08 2022] DELETE /identity/v3/projects/b9a0b2148b9246fe999d36f3f41d5c8d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:09.047481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.047784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.048103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.065879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:09.067671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:09.101074 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:09.128089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1279/2588] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:09.136705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.136940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.137226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.155566 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:09.157072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:09.165113 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1310/2589] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:09.169149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.179368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59647a-a41f-45e4-b713-05ce832eaec6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.184032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.184692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.184983 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.185266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.186008 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:04:09.186324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.187954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.200054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1280/2590] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:04:09.204131 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.213184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea872e2e-0966-4001-9da1-2189ba99711f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.217739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.218386 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.218677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.218962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.219656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.221590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.247671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1311/2591] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:09 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:09.251535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.262254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0078e7-fc47-472c-9332-84493d7832d9 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.318944 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.319611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.319932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.320220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.320907 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.322802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.342968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1281/2592] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:09 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:09.347203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.361600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99205399-078e-44b3-93f6-866374fbd54c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.367545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.368254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.368548 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.368834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.369588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.371214 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.382023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1312/2593] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:04:09.386466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.396828 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105ab5e-c456-427d-88b2-c1dd6a1cf04d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.401340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.402001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.402295 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.402578 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.403315 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.404941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7eac6-24bf-4565-8878-a8eda7689782 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.418436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1282/2594] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:09.419775 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.431487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113be2ba-2204-46ec-b22c-7a49e82024b8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.436500 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.437395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.437685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.438001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] PATH_INFO: `/v3/projects/cbe0605c62d04bed8edf14047746e765/users/72aba3691e2e4b40b6643c8318caa9d6/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.452473 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe0605c62d04bed8edf14047746e765, user_id=72aba3691e2e4b40b6643c8318caa9d6, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.455123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.467686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1313/2595] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:09 2022] PUT /identity/v3/projects/cbe0605c62d04bed8edf14047746e765/users/72aba3691e2e4b40b6643c8318caa9d6/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:04:09.472780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e063e7-78b9-47d3-ba21-4b5be28e075e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.473841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e063e7-78b9-47d3-ba21-4b5be28e075e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.474362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e063e7-78b9-47d3-ba21-4b5be28e075e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.492643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-d6e063e7-78b9-47d3-ba21-4b5be28e075e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:09.493844 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d6e063e7-78b9-47d3-ba21-4b5be28e075e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:09.552680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1283/2596] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:09.562138 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:09.563805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:09.583483 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:09.623838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e26a4-2394-4ef2-8e15-5ff9ba269b83 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:09.627450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:09.628129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:09.628419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:09.628698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:09.638364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:09.639801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:09.650089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1314/2597] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:13.629719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:13.630243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:13.630342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:13.659928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] MFA Rules not processed for user `72aba3691e2e4b40b6643c8318caa9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:13.662250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:13.699598 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:13.734365 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1284/2598] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:04:13.738846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:13.749044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d438790-9585-42a5-a1dd-70724c157fa3 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:13.753832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:13.754805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:13.755101 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:13.755374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:13.756116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:13.758028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:13.785978 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1315/2599] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:04:13 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:04:13.789843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:13.802586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd3bfb-7e19-4550-a5e8-9468ba36d4c5 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:13.863299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:13.864041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:13.864726 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:13.864850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:13.865797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:13.867954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:13.894198 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1285/2600] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:04:13 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:04:13.898239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:13.909091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb424a-959c-4229-904d-db6ea23d167d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:13.914574 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:13.915254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:13.915546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:13.915853 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:13.916599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:13.918248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:13.929656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1316/2601] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:13.933612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:13.944924 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2dfc07-3cb3-4abf-a49c-b3a74a228b07 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:13.952977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:13.953901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:13.954143 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:13.954423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] PATH_INFO: `/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740/users/232a52cf45c740cbb64fb25a15be9f22/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:13.975901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24bbb97ea3f4c1684ae57b2d2314740, user_id=232a52cf45c740cbb64fb25a15be9f22, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:13.979661 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:13.996571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1286/2602] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:13 2022] PUT /identity/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740/users/232a52cf45c740cbb64fb25a15be9f22/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:04:14.009069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:14.019462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c77c8e5-6e0c-4f11-b0a9-74db9172df05 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:14.078610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:14.079418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:14.080032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:14.080180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:14.080914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:14.083218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fd9d5-d4ab-4d8d-9286-350ae5949080 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:14.096904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1317/2603] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:04:14.098255 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:14.109107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70e0fe-8a5f-47fc-902b-1491b243e2dc admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:14.114149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:14.115137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:14.115914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:14.116055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] PATH_INFO: `/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740/users/232a52cf45c740cbb64fb25a15be9f22/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:14.118069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a24bbb97ea3f4c1684ae57b2d2314740, user_id=232a52cf45c740cbb64fb25a15be9f22, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:14.120795 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:14.132341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1287/2604] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:04:14 2022] PUT /identity/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740/users/232a52cf45c740cbb64fb25a15be9f22/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:04:18.670676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dba3c-3a98-4b7b-8509-715d01cb515e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:18.671132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dba3c-3a98-4b7b-8509-715d01cb515e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:18.671256 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dba3c-3a98-4b7b-8509-715d01cb515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:18.717738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7e5dba3c-3a98-4b7b-8509-715d01cb515e None None] MFA Rules not processed for user `232a52cf45c740cbb64fb25a15be9f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:18.719008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e5dba3c-3a98-4b7b-8509-715d01cb515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:18.795793 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1318/2605] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:18.810250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:18.812070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:18.900202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89388227-5adf-4799-a437-6eec12c6745a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:18.905638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:18.906305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:18.906597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:18.906882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:18.932994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:18.971378 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:18.973175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:18.982963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1288/2606] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:19.914843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95acbb6-a688-4060-b0e8-9d4f9e4eadb4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:19.915343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95acbb6-a688-4060-b0e8-9d4f9e4eadb4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:19.915450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95acbb6-a688-4060-b0e8-9d4f9e4eadb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:19.927914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:19.935411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d95acbb6-a688-4060-b0e8-9d4f9e4eadb4 None None] MFA Rules not processed for user `72aba3691e2e4b40b6643c8318caa9d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:19.936881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d95acbb6-a688-4060-b0e8-9d4f9e4eadb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:19.944249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1319/2607] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:28.978370 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=16 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:31.527577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:04:31.529211 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:31.586313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d32a8-5e47-48c4-9445-836a8f9aa463 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:04:31.590171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:04:31.590811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:31.591098 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:31.591373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:31.593173 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:31.649581 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:04:31.651035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4a57-533c-4ff7-84a3-82450055627f service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:04:31.659786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1289/2608] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:04:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:04:41.655717 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=48 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:47.862680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:47.863226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:47.863226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:47.881331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] MFA Rules not processed for user `3acb774848a74e74a5886ee681ed205c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:47.882536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:47.890634 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1320/2609] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:04:57.876199 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:04:58.566958 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:04:58.567450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:04:58.567552 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:04:58.584724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] MFA Rules not processed for user `3eb0459bdac1456a900046e264835a6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:04:58.585917 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:04:58.641540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1290/2610] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:05:08.633774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=25 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:05:10.733226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:10.735517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:10.792044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b9030-da29-4b5a-b34a-9403a15f04e1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:10.795630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:10.796329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:10.797213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:10.797577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:10.799401 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:10.868679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:10.870731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b644d96-4c4d-428a-9299-7e82e48cfee8 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:10.882319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1321/2611] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:12.695207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:12.705597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929a4f7-2d99-4026-9331-668d13f07c51 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:12.709234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:12.709882 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:12.710168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:12.710475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:12.720377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:12.722443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:12.732021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1291/2612] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:20.877509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=50 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:05:22.727900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:05:28.861613 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:28.862295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:28.862295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:28.880259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:05:28.881359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:28.937991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1322/2613] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:05:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:05:28.983560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:28.985007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:29.054409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d478d-39fe-461c-a223-85341cb4e372 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:29.059333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:29.060016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:29.060304 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:29.060579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:29.069208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:29.070780 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:29.078600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1292/2614] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:05:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:05:37.031835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:37.045129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3492-5bfb-4c95-8e79-7ab2465a66be None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:37.049144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:37.049887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:37.050188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:37.050490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:37.052436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:37.067013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:37.068581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2911c633-72fd-4ccd-85ab-0bdaba53eaaa service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:37.080996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1323/2615] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:38.278168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:38.288694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ffcc8-3a30-4614-b1e1-9ad2c769fc86 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:38.292362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:38.293121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:38.293414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:38.293694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:38.295559 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:38.310313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:38.311928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da1673-95f1-4467-8a8e-e1b85cdda446 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:38.324579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1293/2616] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:43.972863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:43.973431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:43.973593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:43.975250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1324/2617] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:05:43.982013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0fe4-7eb7-42b6-a952-4e30b0a52694 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:43.982436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0fe4-7eb7-42b6-a952-4e30b0a52694 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:43.982633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f0fe4-7eb7-42b6-a952-4e30b0a52694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.001931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-696f0fe4-7eb7-42b6-a952-4e30b0a52694 None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:05:44.003812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-696f0fe4-7eb7-42b6-a952-4e30b0a52694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:44.013934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1294/2618] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:05:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:05:44.025956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:44.027639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:44.087408 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951c68a-324b-4076-bac1-ff8c7097e49c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:44.091350 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:44.092049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.092337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.092617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.103836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:44.105290 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5ec87-d3d0-430b-a1d5-e616ce5f03ad service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:44.116629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1325/2619] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:44.183443 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf1f9-2ebe-4f54-8f54-1422ed984c67 None None] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.183735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf1f9-2ebe-4f54-8f54-1422ed984c67 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.184054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf1f9-2ebe-4f54-8f54-1422ed984c67 None None] PATH_INFO: `/` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.185463 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1295/2620] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:05:44.190869 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a2c2-622d-45f9-9e92-eb3d0773455d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.191261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a2c2-622d-45f9-9e92-eb3d0773455d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.191557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a2c2-622d-45f9-9e92-eb3d0773455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.230321 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e4c8a2c2-622d-45f9-9e92-eb3d0773455d None None] MFA Rules not processed for user `05e67d8adf624d458533c80507374d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:05:44.231516 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e4c8a2c2-622d-45f9-9e92-eb3d0773455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:44.238747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1326/2621] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:05:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:05:44.250678 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bf1f9-2ebe-4f54-8f54-1422ed984c67 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:44.262483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bf1f9-2ebe-4f54-8f54-1422ed984c67 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:44.265973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:44.266608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.266897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.267179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.274715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:44.276169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d1a3e-8af4-4367-ba6d-440ef651a575 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:44.283753 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1296/2622] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:44.364901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4dc1d-737a-409f-bf50-8beac7ca3246 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.365189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4dc1d-737a-409f-bf50-8beac7ca3246 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.365493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4dc1d-737a-409f-bf50-8beac7ca3246 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.366895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1327/2623] 10.222.0.24 () {54 vars in 921 bytes} [Tue May 24 14:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 14:05:44.372626 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7010-e8d8-4b65-9cde-cf299cb7ff7d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.372919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7010-e8d8-4b65-9cde-cf299cb7ff7d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.373196 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb7010-e8d8-4b65-9cde-cf299cb7ff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.391659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-04eb7010-e8d8-4b65-9cde-cf299cb7ff7d None None] MFA Rules not processed for user `5bb056b4578442038d33d4fdf9de9584`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:05:44.392993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-04eb7010-e8d8-4b65-9cde-cf299cb7ff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:44.402348 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1297/2624] 10.222.0.24 () {58 vars in 1018 bytes} [Tue May 24 14:05:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:05:44.412899 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4dc1d-737a-409f-bf50-8beac7ca3246 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:44.423838 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4dc1d-737a-409f-bf50-8beac7ca3246 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:44.428104 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:44.428741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:44.429033 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:44.429319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:44.439361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:44.440824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:44.452210 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1328/2625] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:05:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:05:45.993533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:45.994119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:45.994119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:46.014605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:05:46.015812 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:05:46.024213 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1298/2626] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:05:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:05:46.037491 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:05:46.049329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca68025-bf1b-406b-84a3-e671904b371d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:05:46.055264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:05:46.055941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:05:46.056227 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:05:46.056507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:05:46.065481 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:05:46.067867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c82402c-5e72-4d24-97c0-acd2caf6872e service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:05:46.077007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1329/2627] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:05:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:05:56.009107 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=53 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:05:56.072986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=78 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:06:08.372116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:06:08.382079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad4491-e08a-464e-9929-94cf9c827ee1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:06:08.386638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:06:08.387696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:06:08.388253 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:06:08.388642 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:06:08.390489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:06:08.400871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:06:08.402325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:06:08.410468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1299/2628] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:06:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:06:10.601152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:06:10.601659 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:06:10.601782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:06:10.619415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:06:10.620885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:06:10.627905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1330/2629] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 14:06:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:06:10.640522 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:06:10.652420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0178-0dd7-4b0f-bdef-c18ee437bbbd service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:06:10.657341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:06:10.657976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:06:10.658262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:06:10.658541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:06:10.666926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:06:10.668523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:06:10.676328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1300/2630] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:06:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:06:20.614153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:06:20.673142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=12 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:07.433912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:07.435277 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:07.446520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e37c0a-6f40-4811-a6fe-454459f58e5c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:07.450068 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:07.450702 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:07.451002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:07.451276 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:07.452990 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:07.485446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:07.486874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:07.505550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1331/2631] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:12.103897 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:12.106132 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:12.124039 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef526-9738-4758-af81-1b6e01fbb7b3 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:12.130331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:12.130994 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:12.131284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:12.131567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:12.133474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:12.146708 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:12.148352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:12.160335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1301/2632] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:15.219253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:15.233072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e8c1c-01ec-45d6-b7e3-9ccccbc341f4 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:15.240671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:15.241643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:15.242086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:15.242530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:15.245346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:15.258830 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:15.261453 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:15.273542 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1332/2633] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:17.029075 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:17.038500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2100a-1ad8-4101-9c89-f25026a34e77 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:17.042754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:17.043549 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:17.043886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:17.044148 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:17.046017 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:17.109883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:17.111363 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:17.122444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1302/2634] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:21.516615 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:21.518460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:21.624120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:21.626722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:21.651278 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfabf85-0480-4716-8ab8-c7de7c88e6e5 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:21.652141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdec54-10ff-4bb3-81f9-c15ca7b512d1 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:21.657925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:21.658246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:21.660422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:21.660583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:21.660724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:21.660967 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:21.661106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] PATH_INFO: `/v3/users/3eb0459bdac1456a900046e264835a6c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:21.661418 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:21.662293 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb0459bdac1456a900046e264835a6c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:21.663595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:21.664655 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:21.681979 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:21.684351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:21.699675 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1333/2635] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:21.756105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb0459bdac1456a900046e264835a6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:21.758352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb0459bdac1456a900046e264835a6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:21.772187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb0459bdac1456a900046e264835a6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:21.772883 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb0459bdac1456a900046e264835a6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:21.780465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b08533be-cfa5-4789-86c8-beb5e621e652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb0459bdac1456a900046e264835a6c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:21.809299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1303/2636] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:21 2022] DELETE /identity/v3/users/3eb0459bdac1456a900046e264835a6c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:21.998403 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.009612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42fe33-6df4-4bf0-a448-d987c7c1d426 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.079442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.080478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.080833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.081185 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] PATH_INFO: `/v3/projects/3db44af3db3441b7a0c5eb524eb35806` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.084631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db44af3db3441b7a0c5eb524eb35806)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.086584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.131519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:22.142419 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db44af3db3441b7a0c5eb524eb35806', 'request_id': 'req-b1d81a1e-fca7-4766-8627-e3a4cb09f733'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:22.148943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db44af3db3441b7a0c5eb524eb35806', 'request_id': 'req-b1d81a1e-fca7-4766-8627-e3a4cb09f733'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:22.155456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db44af3db3441b7a0c5eb524eb35806', 'request_id': 'req-b1d81a1e-fca7-4766-8627-e3a4cb09f733'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:22.171003 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] The token cache is being invalidate because project 3db44af3db3441b7a0c5eb524eb35806 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:22.171400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:22.172055 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b1d81a1e-fca7-4766-8627-e3a4cb09f733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:22.173780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1334/2637] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:21 2022] DELETE /identity/v3/projects/3db44af3db3441b7a0c5eb524eb35806 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:22.178962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.178962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.179259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.197171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:22.198381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:22.250223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:22.266145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1304/2638] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:22.274662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.275968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.276252 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.298374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:22.299587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:22.308191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1335/2639] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:22.312499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.323015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b527c-20e6-410e-9964-2595ee6c4110 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.328142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.328957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.329302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.329643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.330452 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:22.330826 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.332607 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.345690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1305/2640] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:22.349989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.360154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be54b6-5b08-4de9-ab80-c8bf789ac82c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.365958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.366907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.367198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.367543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.368300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.370953 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.397091 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1336/2641] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:22.401296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.411554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae206a5-683f-4d2e-bce2-ca3baeb6ad8b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.472406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.473091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.473377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.473656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.474364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.476307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.500189 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1306/2642] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:22.506065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.527585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745e725-28ea-421c-bbd4-a7131c8e9bc1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.532416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.533094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.533383 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.533661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.534395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.536032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.547317 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1337/2643] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:22.551500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.562686 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ce861-6c1e-459c-b864-6ddf09e3846e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.567357 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.568079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.568483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.568736 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.569600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.571241 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab6aa04-8bde-4c93-9fe2-1139269b4784 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.583221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1307/2644] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:22.587614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.598454 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61faa03d-2ec4-43f3-a4f6-071bdee28efd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.616684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.617595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.617881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.618153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] PATH_INFO: `/v3/projects/bd04ca0b4c4342e1ae63704ca0d79af4/users/8d7cb20f41c44e689c2554f36830dac2/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.635769 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd04ca0b4c4342e1ae63704ca0d79af4, user_id=8d7cb20f41c44e689c2554f36830dac2, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.638735 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.658225 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1338/2645] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:22 2022] PUT /identity/v3/projects/bd04ca0b4c4342e1ae63704ca0d79af4/users/8d7cb20f41c44e689c2554f36830dac2/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:22.669810 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.670899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.671374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.706459 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] MFA Rules not processed for user `8d7cb20f41c44e689c2554f36830dac2`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:22.707692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:22.789343 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1308/2646] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:22.793931 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.876127 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb474c1-c2f9-46a2-89fc-9c1781d6d466 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.889780 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:22.890671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:22.890956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:22.891242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:22.891996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:22.894166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:22.931109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1339/2647] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/projects => generated 371 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:22.932524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:22.943431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b68d2-039d-4714-af3e-241bc4ce855c None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:22.995692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:23.013340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.014608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.014608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.014831 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.015592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.017630 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.040599 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1309/2648] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:22 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:23.045092 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:23.056579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db4d91-8639-481d-ba78-fe1e805fd60d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:23.062397 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.063112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.063412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.063749 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.064730 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.066368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.081692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1340/2649] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:23.086170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:23.097899 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17220898-c7e7-4b99-963f-697bcfd7f02e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:23.102715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.104323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.104616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.104906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] PATH_INFO: `/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb/users/c213dd0847ab4ea2888b9916d86ce9ef/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.124192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f71537b7b6c4083bc5e32e9d565fdbb, user_id=c213dd0847ab4ea2888b9916d86ce9ef, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.126909 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.141932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1310/2650] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:23 2022] PUT /identity/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb/users/c213dd0847ab4ea2888b9916d86ce9ef/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:23.145597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:23.156216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4577bc2d-297c-4dde-b26c-9055765a2a41 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:23.195423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:23.235348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.236311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.236977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.236977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.237976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.239928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099732d3-841e-4fde-a8ad-8aa7fb4a166e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.251709 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1341/2651] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:23.255666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:23.269449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4014a-74a2-4d99-a9d0-0bbed577312f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:23.275941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.277093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.277557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.277990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] PATH_INFO: `/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb/users/c213dd0847ab4ea2888b9916d86ce9ef/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.280269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f71537b7b6c4083bc5e32e9d565fdbb, user_id=c213dd0847ab4ea2888b9916d86ce9ef, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.283238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.298485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1311/2652] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:23 2022] PUT /identity/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb/users/c213dd0847ab4ea2888b9916d86ce9ef/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:23.310981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.311303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.311563 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.344789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] MFA Rules not processed for user `c213dd0847ab4ea2888b9916d86ce9ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:23.344789 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:23.434945 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1342/2653] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:23.448426 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:23.450635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:23.516331 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187de39-d95c-4db0-9332-fbd59f35bac5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:23.520201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:23.520906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:23.521200 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:23.521482 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:23.532587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:23.534062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c27b48-7dab-44f8-b093-97fb04931c00 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:23.548366 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1312/2654] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:07:24.048384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:24.133090 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf2daa-0cc3-40d8-896f-4922222b25bf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:24.138541 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:24.139432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:24.139724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:24.140022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] PATH_INFO: `/v3/users/8d7cb20f41c44e689c2554f36830dac2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:24.141242 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d7cb20f41c44e689c2554f36830dac2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:24.143186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:24.192054 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d7cb20f41c44e689c2554f36830dac2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.199449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d7cb20f41c44e689c2554f36830dac2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.201901 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:24.214100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d7cb20f41c44e689c2554f36830dac2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.214951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d7cb20f41c44e689c2554f36830dac2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.215692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d7cb20f41c44e689c2554f36830dac2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.229469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1343/2655] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:24 2022] DELETE /identity/v3/users/8d7cb20f41c44e689c2554f36830dac2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:24.234126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76992-031e-4442-85f5-607389daba22 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:24.234636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76992-031e-4442-85f5-607389daba22 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:24.234843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec76992-031e-4442-85f5-607389daba22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:24.253985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-8ec76992-031e-4442-85f5-607389daba22 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:24.255507 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ec76992-031e-4442-85f5-607389daba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:24.317600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1313/2656] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:24.328777 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:24.330345 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:24.398239 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b3e3bc-a6d5-418b-9518-af8f9a26d994 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:24.402436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:24.403095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:24.403387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:24.403661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:24.413223 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:24.414673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:24.425118 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1344/2657] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:24.797625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76992-031e-4442-85f5-607389daba22 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:24.799191 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8ec76992-031e-4442-85f5-607389daba22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:24.819397 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec76992-031e-4442-85f5-607389daba22 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:24.836172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:24.838025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:24.838321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:24.838605 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] PATH_INFO: `/v3/projects/bd04ca0b4c4342e1ae63704ca0d79af4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:24.840181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd04ca0b4c4342e1ae63704ca0d79af4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:24.842395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:24.891953 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:24.900667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd04ca0b4c4342e1ae63704ca0d79af4', 'request_id': 'req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.909215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd04ca0b4c4342e1ae63704ca0d79af4', 'request_id': 'req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.916135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd04ca0b4c4342e1ae63704ca0d79af4', 'request_id': 'req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.934820 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] The token cache is being invalidate because project bd04ca0b4c4342e1ae63704ca0d79af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:24.935121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.936567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:24.938086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1314/2658] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:24 2022] DELETE /identity/v3/projects/bd04ca0b4c4342e1ae63704ca0d79af4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:24.941879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:24.952657 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ced400-7466-4fdf-b530-43ada4e5bfa9 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.010226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.011114 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.011409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.011686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] PATH_INFO: `/v3/users/c213dd0847ab4ea2888b9916d86ce9ef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.012943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c213dd0847ab4ea2888b9916d86ce9ef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.014867 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.067981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c213dd0847ab4ea2888b9916d86ce9ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.076970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c213dd0847ab4ea2888b9916d86ce9ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.094746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c213dd0847ab4ea2888b9916d86ce9ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.095612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c213dd0847ab4ea2888b9916d86ce9ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.096393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f28bceb0-e701-4eca-982c-06b1cb95453e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c213dd0847ab4ea2888b9916d86ce9ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.109763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:25.119616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1345/2659] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:24 2022] DELETE /identity/v3/users/c213dd0847ab4ea2888b9916d86ce9ef => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:25.435853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.437971 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:25.452274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c7816-c7c3-4b5e-90f9-ff98bff20dfb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.557919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.558869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.559183 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.559468 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] PATH_INFO: `/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.560700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f71537b7b6c4083bc5e32e9d565fdbb)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.562688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.609821 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f71537b7b6c4083bc5e32e9d565fdbb', 'request_id': 'req-f20dba83-6def-41b2-9277-c7e29ede12b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.616136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f71537b7b6c4083bc5e32e9d565fdbb', 'request_id': 'req-f20dba83-6def-41b2-9277-c7e29ede12b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.622186 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f71537b7b6c4083bc5e32e9d565fdbb', 'request_id': 'req-f20dba83-6def-41b2-9277-c7e29ede12b0'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.636436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] The token cache is being invalidate because project 8f71537b7b6c4083bc5e32e9d565fdbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:25.636640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.637313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f20dba83-6def-41b2-9277-c7e29ede12b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:25.638860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1315/2660] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:25 2022] DELETE /identity/v3/projects/8f71537b7b6c4083bc5e32e9d565fdbb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:25.644472 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.644873 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.645249 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.664683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:25.666038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:25.729565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1346/2661] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:25.736739 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.737031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.737314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.757447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:25.758666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:25.767119 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1316/2662] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:25.771255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.781452 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e645b-e8b0-479a-b074-0447e132efd7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.785948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.786591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.786876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.787153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.787910 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:25.788220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.789801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.801897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1347/2663] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:25.806155 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.815437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab76a52-c901-403a-bc4d-ce2d75e4007d None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.820117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.820768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.821054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.821344 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.822113 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.825084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.858170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1317/2664] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:25 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:25.862158 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.873677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897af25-3e47-4ef7-86a3-61bd5b92bdc3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.936677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.937362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.937654 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.937867 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.937982 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.938677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.939538 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:25.940674 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.965358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1348/2665] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:25 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:25.966513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:25.974409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:25.977012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37a37f-6b42-47ba-8357-f560e808576c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:25.983655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:25.984302 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:25.984596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:25.984884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:25.985658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:25.987509 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:25.997813 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1349/2666] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:26.001975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.003195 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b1d71-a81e-4c09-b3da-1e913677a2bf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.007412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.008087 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.008373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.008647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.010677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:26.011561 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fafafb-1b69-4df8-a2ed-014ba2c54fff admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.016493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.017139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.017432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.017718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.018448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.020163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.022454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.023896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df178020-4540-4868-8f33-7eacd9fbdb0e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.034844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1350/2667] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:26.036308 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.039995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1318/2668] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:07:26.046756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac8e0f-80f3-4985-8f76-18612c6f8982 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.051436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.052352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.052641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.052924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] PATH_INFO: `/v3/projects/468ed66ab8ea47f9a6229ac7d49b7b61/users/8dc64a19b3cb4df2a750243f35fcf256/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.068660 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=468ed66ab8ea47f9a6229ac7d49b7b61, user_id=8dc64a19b3cb4df2a750243f35fcf256, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.071315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.084546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1351/2669] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:26 2022] PUT /identity/v3/projects/468ed66ab8ea47f9a6229ac7d49b7b61/users/8dc64a19b3cb4df2a750243f35fcf256/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:26.095282 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.095580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.095901 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.122648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] MFA Rules not processed for user `8dc64a19b3cb4df2a750243f35fcf256`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:26.124144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:26.203521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1319/2670] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:26.208123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.240484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:26.277926 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005504-67b8-480d-8f2c-7191cfa99d9a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.283415 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.285261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.285261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.285261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.285779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.287799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.325520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1352/2671] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:26 2022] POST /identity/v3/projects => generated 369 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:26.330291 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.342324 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc8f89-5b8b-4b76-8e03-53e6d8753125 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.405766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.406487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.406776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.407057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.407893 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.409913 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.446231 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1320/2672] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:26 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:07:26.450798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.461952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f246a-d4f4-47e1-b8fb-2cad9a889a2a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.467807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.468799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.469823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.470125 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.471069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.473283 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.485890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1353/2673] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:26.490262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.503194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af841ba8-f326-4cd4-8094-db52ea024fc5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.508799 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.510207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.510725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.511097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] PATH_INFO: `/v3/projects/9b5b933bd41e45c69287af75deee01b8/users/84e2b85cdd8846c1a98b7767aae1ee22/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.528588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5b933bd41e45c69287af75deee01b8, user_id=84e2b85cdd8846c1a98b7767aae1ee22, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.531533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.546890 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1321/2674] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:26 2022] PUT /identity/v3/projects/9b5b933bd41e45c69287af75deee01b8/users/84e2b85cdd8846c1a98b7767aae1ee22/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:26.551218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.563162 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a1852c-a503-4332-9047-db1fe9219433 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.630111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.630795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.631106 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.631364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.632167 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.633801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76024306-2403-478a-8088-2a0c5789d34e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.647805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1354/2675] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:26.649058 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.662660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8faa8c-55ee-4737-98ad-fbc00d5002df admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.671328 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.673719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.674121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.674521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] PATH_INFO: `/v3/projects/9b5b933bd41e45c69287af75deee01b8/users/84e2b85cdd8846c1a98b7767aae1ee22/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.677025 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b5b933bd41e45c69287af75deee01b8, user_id=84e2b85cdd8846c1a98b7767aae1ee22, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.680162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.700879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1322/2676] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:26 2022] PUT /identity/v3/projects/9b5b933bd41e45c69287af75deee01b8/users/84e2b85cdd8846c1a98b7767aae1ee22/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:26.706247 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.706623 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.706976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.740051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] MFA Rules not processed for user `84e2b85cdd8846c1a98b7767aae1ee22`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:26.741520 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:26.818095 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1355/2677] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:26.830474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:26.832212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:26.847845 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:26.907250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863c061-7d9c-43c7-9f6e-1dcbf1b6aded admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:26.912174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:26.912961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:26.913271 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:26.913555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:26.925808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:26.927673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:26.939207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1323/2678] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:27.012663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:27.023927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd3663-98f3-4fe2-87f2-3a2a2ad6efc8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:27.030155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:27.030890 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:27.031184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:27.031462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:27.059310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:27.106236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:27.107752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:27.118526 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1356/2679] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:28.435916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:28.503975 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0c8c-afd6-4a9e-8a7b-1f0da790bfb6 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:28.508837 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:28.511115 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:28.511862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:28.512027 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:28.515384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:28.600179 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:28.602320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:28.616187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1324/2680] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:30.126099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:30.128238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:30.146722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3be38-f55d-472e-9e3e-fbc7aa436c6c service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:30.153089 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:30.154357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:30.154872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:30.155479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] PATH_INFO: `/v3/users/b3560616f4ec496f816602f9c73b69ef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:30.189121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3560616f4ec496f816602f9c73b69ef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:30.191173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:30.267311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3560616f4ec496f816602f9c73b69ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.277263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3560616f4ec496f816602f9c73b69ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.292272 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3560616f4ec496f816602f9c73b69ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.293154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3560616f4ec496f816602f9c73b69ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.293774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3560616f4ec496f816602f9c73b69ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.321280 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1357/2681] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:30 2022] DELETE /identity/v3/users/b3560616f4ec496f816602f9c73b69ef => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:30.567134 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:30.589351 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457defe5-5a83-483e-991d-7eb716c8f307 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:30.741320 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:30.741919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:30.742214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:30.742502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] PATH_INFO: `/v3/projects/d3ebc40e12384ce4bb168690cfd5a27a` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:30.754471 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:30.757943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ebc40e12384ce4bb168690cfd5a27a)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:30.760657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:30.826692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:30.828248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:30.842938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ebc40e12384ce4bb168690cfd5a27a', 'request_id': 'req-de3ba05d-c4f5-469d-9674-6f0f1659f364'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.844384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01010ce5-092a-46a3-9a4b-8bc18c70cb55 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:30.855147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ebc40e12384ce4bb168690cfd5a27a', 'request_id': 'req-de3ba05d-c4f5-469d-9674-6f0f1659f364'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.862748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ebc40e12384ce4bb168690cfd5a27a', 'request_id': 'req-de3ba05d-c4f5-469d-9674-6f0f1659f364'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.884353 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] The token cache is being invalidate because project d3ebc40e12384ce4bb168690cfd5a27a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:30.884680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.886660 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-de3ba05d-c4f5-469d-9674-6f0f1659f364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:30.888666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1325/2682] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:30 2022] DELETE /identity/v3/projects/d3ebc40e12384ce4bb168690cfd5a27a => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:30.893281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631e8ff-cf82-42ea-8e9f-63fd5f0049d2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:30.893707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631e8ff-cf82-42ea-8e9f-63fd5f0049d2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:30.894010 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631e8ff-cf82-42ea-8e9f-63fd5f0049d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:30.923848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1631e8ff-cf82-42ea-8e9f-63fd5f0049d2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:30.925585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1631e8ff-cf82-42ea-8e9f-63fd5f0049d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:30.925968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:30.926868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:30.927153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:30.927462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] PATH_INFO: `/v3/users/8dc64a19b3cb4df2a750243f35fcf256` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:30.928766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dc64a19b3cb4df2a750243f35fcf256)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:30.931798 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:30.939224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1326/2683] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:30.946142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:30.947099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:30.947390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:30.969711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:30.971580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:30.984707 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1327/2684] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:30.985372 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:30.986151 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.000214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdcce6-fc71-4564-93cf-7c4f3701462e None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.008415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.008839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.009667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.009965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.010850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:31.011092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.013014 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.026507 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dc64a19b3cb4df2a750243f35fcf256'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:31.027865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1328/2685] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:31.033084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.047598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dc64a19b3cb4df2a750243f35fcf256'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:31.052657 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc72e6-75ec-49d0-aff5-0c044a34ba0f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.060587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.061640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.061936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.062223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.064093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.064477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dc64a19b3cb4df2a750243f35fcf256'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:31.065508 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dc64a19b3cb4df2a750243f35fcf256'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:31.066611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-12883521-9254-486f-ac00-9fe7347c7469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dc64a19b3cb4df2a750243f35fcf256'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:31.066752 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.084848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1358/2686] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:30 2022] DELETE /identity/v3/users/8dc64a19b3cb4df2a750243f35fcf256 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:31.090194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.090630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.090808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.107269 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1329/2687] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:31 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:07:31.111666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.115133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:31.116461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.121500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.185611 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1359/2688] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:31.192564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.194012 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.207942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68eb868-b4ed-4235-bf28-146d60d5fe67 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.214759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.215570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.215885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.216174 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.217085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.219178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.247931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1330/2689] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:31 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:31.249713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.259952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcfe81-6d57-439f-8825-c2d5800ac7de admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.260711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004ea07-2762-481d-bb23-45b368b1c642 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.264721 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.265336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.265631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.265914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.266080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.266651 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.266875 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.267173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.268129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.268317 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.279533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1331/2690] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:31.284024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.284024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.285095 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.294351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1360/2691] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:07:31.298466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aead9e-87c9-4448-99af-b7451b1cc932 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.304284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.305102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.305465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.305675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.306577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.308433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844da8-e9cb-4e02-a221-9c276ec38242 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.323030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1332/2692] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:31.323705 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.335629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da7aef-afe8-49c1-bb03-fb1f624c07bf service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.342786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.343841 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.344170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.344528 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] PATH_INFO: `/v3/projects/e436feb329204699a78d749db135acce/users/54fce865bd6842c19583b3bfe607fc32/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.369348 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e436feb329204699a78d749db135acce, user_id=54fce865bd6842c19583b3bfe607fc32, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.372609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.393598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1361/2693] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:31 2022] PUT /identity/v3/projects/e436feb329204699a78d749db135acce/users/54fce865bd6842c19583b3bfe607fc32/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:31.395003 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.395303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.395586 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.411556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:31.417894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:31.420130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.490728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1333/2694] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:31.503070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.505220 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.554306 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:31.593631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf1535-e8d8-43aa-8fba-584ad8dbb3ec admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:31.598151 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:31.598835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:31.599123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:31.599404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:31.612661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:31.614758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:31.627905 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1362/2695] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:31.955616 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:31.957214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:31.997341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de7033-e410-4b27-9486-e657246c8419 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:32.007641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:32.008576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:32.008872 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:32.009225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] PATH_INFO: `/v3/projects/468ed66ab8ea47f9a6229ac7d49b7b61` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:32.012745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468ed66ab8ea47f9a6229ac7d49b7b61)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:32.015156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:32.135567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468ed66ab8ea47f9a6229ac7d49b7b61', 'request_id': 'req-215bfb0c-81be-4b8c-a1a5-836f729a0e26'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.147702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468ed66ab8ea47f9a6229ac7d49b7b61', 'request_id': 'req-215bfb0c-81be-4b8c-a1a5-836f729a0e26'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.169762 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468ed66ab8ea47f9a6229ac7d49b7b61', 'request_id': 'req-215bfb0c-81be-4b8c-a1a5-836f729a0e26'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.193615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] The token cache is being invalidate because project 468ed66ab8ea47f9a6229ac7d49b7b61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:32.193995 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.194654 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.197307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1334/2696] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:31 2022] DELETE /identity/v3/projects/468ed66ab8ea47f9a6229ac7d49b7b61 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:32.202019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:32.218112 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f08c8a-0117-4c06-93f2-30e426a5804c service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:32.324388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:32.325766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:32.325766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:32.326186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] PATH_INFO: `/v3/users/84e2b85cdd8846c1a98b7767aae1ee22` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:32.332811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e2b85cdd8846c1a98b7767aae1ee22)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:32.335368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:32.417226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e2b85cdd8846c1a98b7767aae1ee22'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.438351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e2b85cdd8846c1a98b7767aae1ee22'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.457710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e2b85cdd8846c1a98b7767aae1ee22'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.458550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e2b85cdd8846c1a98b7767aae1ee22'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.459466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-246b1186-6ea8-4773-b40c-69afdc2e5ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e2b85cdd8846c1a98b7767aae1ee22'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:32.512544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1363/2697] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:32 2022] DELETE /identity/v3/users/84e2b85cdd8846c1a98b7767aae1ee22 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:33.143521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:33.145807 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:33.164808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215bfb0c-81be-4b8c-a1a5-836f729a0e26 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:33.239613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:33.240633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.240999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.241385 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] PATH_INFO: `/v3/projects/9b5b933bd41e45c69287af75deee01b8` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.242687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b5b933bd41e45c69287af75deee01b8)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:33.246032 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:33.274511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:33.317498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b5b933bd41e45c69287af75deee01b8', 'request_id': 'req-e23ea84e-ba24-4ffa-a388-dce24fa4d021'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:33.329145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b5b933bd41e45c69287af75deee01b8', 'request_id': 'req-e23ea84e-ba24-4ffa-a388-dce24fa4d021'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:33.338610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b5b933bd41e45c69287af75deee01b8', 'request_id': 'req-e23ea84e-ba24-4ffa-a388-dce24fa4d021'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:33.366136 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] The token cache is being invalidate because project 9b5b933bd41e45c69287af75deee01b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:33.366802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:33.367773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-e23ea84e-ba24-4ffa-a388-dce24fa4d021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:33.370234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1335/2698] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:33 2022] DELETE /identity/v3/projects/9b5b933bd41e45c69287af75deee01b8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:33.375479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.376123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.376123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.408806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:33.410380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:33.470257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:33.501864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1364/2699] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:33.510527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.511172 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.511627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.544407 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:33.548220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:33.562055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1336/2700] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:33.566377 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:33.592849 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431cef18-6bd6-4691-a9c2-af58168eef2c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:33.600810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:33.601495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.601935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.602221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.603060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:33.604039 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:33.605908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:33.622216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1365/2701] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:33.626268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:33.636720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a0b79-3dd1-4dca-94d2-b30574d02136 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:33.642490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:33.643171 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.643454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.643735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.644476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:33.646528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:33.687989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1337/2702] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:33 2022] POST /identity/v3/projects => generated 399 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:33.692032 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:33.706822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba07ea-71e8-40ca-9ded-59a109caf171 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:33.937461 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:33.938150 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:33.938445 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:33.938715 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:33.939410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:33.941389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.007695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1366/2703] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:33 2022] POST /identity/v3/users => generated 300 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:34.012781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.104602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bccd724-2be4-47a3-b9dc-e36224c2c4bb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.109700 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.113222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.113567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.113859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.114698 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.116371 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.138682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1338/2704] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:34.146876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.159537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3f09-e4f2-4a54-aef5-f092f66e2d03 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.174810 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.176413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.176805 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.177120 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.180371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.182585 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d94c3-1731-44e4-85e6-e9c8a95b2a73 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.204783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1367/2705] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:34.208384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.225481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f156d87-2e90-447d-bac0-c717784fe64a admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.237514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.238485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.238720 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.239001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] PATH_INFO: `/v3/projects/0a995f4bf8334009aa96d7a13e374e9c/users/090b41b7b7ac49a7b2fb200de6885628/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.267875 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a995f4bf8334009aa96d7a13e374e9c, user_id=090b41b7b7ac49a7b2fb200de6885628, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.271526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.307178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1339/2706] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:34 2022] PUT /identity/v3/projects/0a995f4bf8334009aa96d7a13e374e9c/users/090b41b7b7ac49a7b2fb200de6885628/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:34.317144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.317470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.317711 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.349602 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] MFA Rules not processed for user `090b41b7b7ac49a7b2fb200de6885628`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:34.350886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:34.413578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:34.442564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1368/2707] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:34.443431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.535055 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa8673-0ffa-4893-a0ab-13bed3b8c941 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.542130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.542946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.543336 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.544356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.545100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.548784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.575701 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:34.598212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.606497 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1340/2708] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:34 2022] POST /identity/v3/projects => generated 397 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:07:34.610994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c66d39-5e06-4410-bf9d-6c2ea16d412f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.691876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.693132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.693495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.693994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.694909 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.698496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.738238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.749441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1369/2709] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:34 2022] POST /identity/v3/users => generated 305 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:34.754444 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d877cbe-247c-43d6-b20e-cf35d8a1c179 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.763128 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.764942 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.765976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.766425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.768389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.770057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.802603 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1341/2710] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:34.805073 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.825595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4c3b8-8eba-4cde-ac64-3364981061ea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:34.835865 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:34.837219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:34.837864 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:34.838502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] PATH_INFO: `/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8/users/80d0f1ae316849cbaa72c03fdf7a0221/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:34.871333 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e36994be6db4613b2bb45a1b1daaeb8, user_id=80d0f1ae316849cbaa72c03fdf7a0221, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:34.874723 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:34.910143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1370/2711] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:34 2022] PUT /identity/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8/users/80d0f1ae316849cbaa72c03fdf7a0221/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:34.914085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:34.928306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c18a4-b8ec-417b-b69a-b4119341fc75 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:35.045916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:35.046615 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:35.046905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:35.047187 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:35.048333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:35.056580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0e7f14-c08f-43c9-bcfc-15a15e3bd7a6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:35.081384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1342/2712] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:35.085803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:35.097438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108fdb-f9b1-493c-9096-ec2a0d4b069b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:35.114083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:35.115061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:35.117754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:35.118053 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] PATH_INFO: `/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8/users/80d0f1ae316849cbaa72c03fdf7a0221/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:35.130868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e36994be6db4613b2bb45a1b1daaeb8, user_id=80d0f1ae316849cbaa72c03fdf7a0221, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:35.138182 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:35.162061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1371/2713] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:35 2022] PUT /identity/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8/users/80d0f1ae316849cbaa72c03fdf7a0221/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:35.174279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:35.174634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:35.174905 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:35.232178 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] MFA Rules not processed for user `80d0f1ae316849cbaa72c03fdf7a0221`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:35.235244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:35.349234 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1343/2714] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:35.360643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:35.362504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:35.523330 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079d1c29-fc92-4c6b-bbff-9d3983d126e4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:35.532773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:35.533825 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:35.534337 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:35.534682 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:35.548952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:35.550435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:35.563087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1372/2715] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:35.684871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:35.709427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072e3ff-27e5-4e7a-87bb-1efdb888bbeb None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:35.718874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:35.719685 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:35.720071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:35.720934 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:35.875541 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:35.879704 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:35.912011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1344/2716] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:37.019218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:37.020792 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:37.060005 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:37.105929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356038-7ec7-49de-929e-01e75255ecfe service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:37.111366 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:37.112718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:37.113270 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:37.113779 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] PATH_INFO: `/v3/users/3acb774848a74e74a5886ee681ed205c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:37.115371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acb774848a74e74a5886ee681ed205c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:37.117630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:37.204107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acb774848a74e74a5886ee681ed205c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.219405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acb774848a74e74a5886ee681ed205c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.235626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acb774848a74e74a5886ee681ed205c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.236353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acb774848a74e74a5886ee681ed205c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.237028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acb774848a74e74a5886ee681ed205c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.259446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1373/2717] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:37 2022] DELETE /identity/v3/users/3acb774848a74e74a5886ee681ed205c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:37.498839 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:37.529918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47bcba-b76f-4332-8644-af1ffffad06b service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:37.542886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:37.629377 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:37.630438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:37.630733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:37.631052 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] PATH_INFO: `/v3/projects/7bb15f1d3e524d67b5493cb483ee8547` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:37.632254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bb15f1d3e524d67b5493cb483ee8547)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:37.635221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:37.721877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bb15f1d3e524d67b5493cb483ee8547', 'request_id': 'req-20225e02-9121-4c73-a365-3bbc4f28fd0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.733053 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bb15f1d3e524d67b5493cb483ee8547', 'request_id': 'req-20225e02-9121-4c73-a365-3bbc4f28fd0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.739873 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bb15f1d3e524d67b5493cb483ee8547', 'request_id': 'req-20225e02-9121-4c73-a365-3bbc4f28fd0f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.757935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] The token cache is being invalidate because project 7bb15f1d3e524d67b5493cb483ee8547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:37.758625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.761318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-20225e02-9121-4c73-a365-3bbc4f28fd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.767786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1345/2718] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:37 2022] DELETE /identity/v3/projects/7bb15f1d3e524d67b5493cb483ee8547 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:37.771751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:37.774237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:37.795645 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9603-36f6-4eff-ab70-238e41be18a4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:37.872051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:37.873045 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:37.873335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:37.873656 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] PATH_INFO: `/v3/users/e0ef9587128f470ab550b38a4b7d4eb1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:37.875173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ef9587128f470ab550b38a4b7d4eb1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:37.877289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:37.972048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ef9587128f470ab550b38a4b7d4eb1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:37.973639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:37.989255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ef9587128f470ab550b38a4b7d4eb1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.015619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ef9587128f470ab550b38a4b7d4eb1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.016343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ef9587128f470ab550b38a4b7d4eb1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.016984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ef9587128f470ab550b38a4b7d4eb1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.038564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1374/2719] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:37 2022] DELETE /identity/v3/users/e0ef9587128f470ab550b38a4b7d4eb1 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:38.096297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:38.096595 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:38.096881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:38.139858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] MFA Rules not processed for user `54fce865bd6842c19583b3bfe607fc32`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:38.141162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:38.227568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1346/2720] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:38.247200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:38.250747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:38.302728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:38.305215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:38.358368 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:38.390332 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de36b86-37c8-4b65-94f8-6a426aac8bf1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:38.401416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:38.401844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:38.402130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:38.402435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:38.412285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:38.413965 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50870f68-26c9-4ef5-9f49-f71326448946 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:38.418532 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da42e35-03f1-4f37-b10b-22cee28bc195 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:38.424373 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:38.424620 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1375/2721] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:38.425332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:38.425625 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:38.425935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] PATH_INFO: `/v3/projects/7acb9a364a7a40bbbaacdd321517b426` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:38.427245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7acb9a364a7a40bbbaacdd321517b426)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:38.429209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:38.483775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7acb9a364a7a40bbbaacdd321517b426', 'request_id': 'req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.491130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7acb9a364a7a40bbbaacdd321517b426', 'request_id': 'req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.502341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7acb9a364a7a40bbbaacdd321517b426', 'request_id': 'req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.520989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] The token cache is being invalidate because project 7acb9a364a7a40bbbaacdd321517b426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:38.521390 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.522356 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:38.524251 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1347/2722] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:38 2022] DELETE /identity/v3/projects/7acb9a364a7a40bbbaacdd321517b426 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:39.795625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:39.796251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:39.796251 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:39.814495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] MFA Rules not processed for user `54fce865bd6842c19583b3bfe607fc32`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:39.815708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:39.872269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1376/2723] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:40.264563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:40.266067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.365274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447ead9-2eaa-4edf-ad7d-9dbf7c19f8d2 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:40.372840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:40.373993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:40.374824 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:40.375338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:40.378773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.395078 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:40.396760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc766e1-3cf8-422d-bf55-360291c7f960 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:40.410066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1348/2724] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:40.565111 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:40.566501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.638529 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:40.646267 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ef71-0c44-4dc9-a623-e7a0246906fc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:40.655580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:40.656665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:40.657197 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:40.657855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] PATH_INFO: `/v3/users/090b41b7b7ac49a7b2fb200de6885628` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:40.659355 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=090b41b7b7ac49a7b2fb200de6885628)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:40.661592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:40.719522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '090b41b7b7ac49a7b2fb200de6885628'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:40.730457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '090b41b7b7ac49a7b2fb200de6885628'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:40.753538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '090b41b7b7ac49a7b2fb200de6885628'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:40.754513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '090b41b7b7ac49a7b2fb200de6885628'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:40.755289 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '090b41b7b7ac49a7b2fb200de6885628'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:40.773759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1377/2725] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:40 2022] DELETE /identity/v3/users/090b41b7b7ac49a7b2fb200de6885628 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:40.779284 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:40.779584 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:40.779966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:40.823045 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:40.826712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.845072 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:40.846582 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.889208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:40.933209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1349/2726] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:07:40.942210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:40.944255 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6328c0-e031-4aa8-bb7d-b329f17b18f1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:40.944955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.950024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:40.950694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:40.950981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:40.951260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:40.955312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:40.968699 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:40.970320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:40.981912 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1378/2727] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:07:41.023060 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a3fd0-9267-46ed-9833-5900735be2ed None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:41.027758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:41.031688 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:41.032080 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:41.032431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:41.049199 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:41.061818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:41.081130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1350/2728] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:41.411752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:41.413930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:41.438708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf84c10-4688-49f9-9e7d-7a606fcd5a5d service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:41.449189 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:41.450358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:41.450688 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:41.450939 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] PATH_INFO: `/v3/projects/0a995f4bf8334009aa96d7a13e374e9c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:41.452997 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a995f4bf8334009aa96d7a13e374e9c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:41.454999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:41.519411 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a995f4bf8334009aa96d7a13e374e9c', 'request_id': 'req-8d8d4596-b7fd-49ff-b524-7c73cf62511d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.527428 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a995f4bf8334009aa96d7a13e374e9c', 'request_id': 'req-8d8d4596-b7fd-49ff-b524-7c73cf62511d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.534893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:41.537739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a995f4bf8334009aa96d7a13e374e9c', 'request_id': 'req-8d8d4596-b7fd-49ff-b524-7c73cf62511d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.557456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] The token cache is being invalidate because project 0a995f4bf8334009aa96d7a13e374e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:41.557754 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.558482 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.563467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1379/2729] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:41 2022] DELETE /identity/v3/projects/0a995f4bf8334009aa96d7a13e374e9c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:41.564180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:41.577063 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee197b-ac31-4e7c-abb9-848c0fd7b69f service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:41.656617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:41.657509 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:41.657797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:41.658077 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] PATH_INFO: `/v3/users/80d0f1ae316849cbaa72c03fdf7a0221` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:41.659297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80d0f1ae316849cbaa72c03fdf7a0221)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:41.661303 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:41.725094 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80d0f1ae316849cbaa72c03fdf7a0221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.725790 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80d0f1ae316849cbaa72c03fdf7a0221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.745514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80d0f1ae316849cbaa72c03fdf7a0221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.746431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80d0f1ae316849cbaa72c03fdf7a0221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.754870 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-ef6c9272-782e-422b-9667-1ce9c1fb5c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80d0f1ae316849cbaa72c03fdf7a0221'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:41.765926 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:41.772530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1351/2730] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:41 2022] DELETE /identity/v3/users/80d0f1ae316849cbaa72c03fdf7a0221 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:42.216601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.218964 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:42.238938 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8d4596-b7fd-49ff-b524-7c73cf62511d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.334371 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.335246 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.335530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.335807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] PATH_INFO: `/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.337023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e36994be6db4613b2bb45a1b1daaeb8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.339533 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.390243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e36994be6db4613b2bb45a1b1daaeb8', 'request_id': 'req-a14b0b8c-e500-4b7c-bb93-f922235d9c57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:42.396671 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e36994be6db4613b2bb45a1b1daaeb8', 'request_id': 'req-a14b0b8c-e500-4b7c-bb93-f922235d9c57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:42.402856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e36994be6db4613b2bb45a1b1daaeb8', 'request_id': 'req-a14b0b8c-e500-4b7c-bb93-f922235d9c57'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:42.416178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] The token cache is being invalidate because project 7e36994be6db4613b2bb45a1b1daaeb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:42.416663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:42.417538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a14b0b8c-e500-4b7c-bb93-f922235d9c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:42.419339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1380/2731] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:42 2022] DELETE /identity/v3/projects/7e36994be6db4613b2bb45a1b1daaeb8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:42.423902 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.424207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.424495 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.464775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:42.466006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:42.530979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1352/2732] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:42.538612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.538908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.539195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.558537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:42.559757 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:42.569008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1381/2733] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:42.573395 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.583422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e76b1e-cbe9-42be-8585-f44ea27614b1 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.588966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.589772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.590161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.590454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.591209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:42.591524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.593168 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.606568 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1353/2734] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:42.610820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.620094 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933774be-5dee-40ca-b83d-f154967833d5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.625046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.625707 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.625998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.626281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.626981 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.628930 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.656404 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1382/2735] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:42 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:42.660571 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.670682 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99111ad0-cd0b-4041-ada2-bf94c02356f5 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.728425 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.729264 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.729551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.729832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.730547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.732515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.754209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1354/2736] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:42 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:42.757969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.772001 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b86085-ae68-4343-98cb-eccb321bbf0d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.777115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.777895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.778186 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.778466 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.779349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.780998 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.792593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1383/2737] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:42.796644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.807691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b2486d-e8bf-4fda-b026-4bf5a065bd48 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.812352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.813013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.813306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.813590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.814321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.815970 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de5759-47df-4edc-880c-57b396ddc117 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.827542 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1355/2738] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:42.831961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:42.842322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5be5e5-797e-4641-b049-bba99ad48304 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:42.846936 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:42.847835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.848132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.848412 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] PATH_INFO: `/v3/projects/e5fa913bada04fcfae0549a6491a48e6/users/df79ea5f32354aa883645c02c751fec4/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.864626 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fa913bada04fcfae0549a6491a48e6, user_id=df79ea5f32354aa883645c02c751fec4, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:42.867338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:42.886686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1384/2739] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:42 2022] PUT /identity/v3/projects/e5fa913bada04fcfae0549a6491a48e6/users/df79ea5f32354aa883645c02c751fec4/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:42.898889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:42.899438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:42.900611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:42.930352 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] MFA Rules not processed for user `df79ea5f32354aa883645c02c751fec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:42.931903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:42.975738 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:43.012885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1356/2740] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:43.017067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.034320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:43.095271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3dbf4-058e-48e0-85a7-d5307a7d9339 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.100393 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.101065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.101616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.101616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.102319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.104658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.133540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1385/2741] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:43 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:07:43.137527 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.150769 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f32f7e-76e7-48e8-a761-9bf23d455bf0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.250449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.251135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.251423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.251703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.252478 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.254691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.287092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1357/2742] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:43 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:43.288368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.300304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1c8d7-8783-4418-8ede-64d581795635 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.305075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.305914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.306961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.307282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.308058 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.309683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.322400 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1386/2743] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:43.326798 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.336965 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6898ae3-9d02-4906-93ec-dd5b2d8c4343 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.341722 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.342597 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.342894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.343193 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] PATH_INFO: `/v3/projects/d34ea1fbde72463b873cdacf7487301b/users/560c0e307180420795cc31b94128fa9c/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.358398 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34ea1fbde72463b873cdacf7487301b, user_id=560c0e307180420795cc31b94128fa9c, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.361116 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.376442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1358/2744] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:43 2022] PUT /identity/v3/projects/d34ea1fbde72463b873cdacf7487301b/users/560c0e307180420795cc31b94128fa9c/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:07:43.380960 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.391934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dc5a-9478-42ad-b686-cf69fc13916f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.456070 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.456766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.457105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.457339 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.458238 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.461710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60032175-ccbf-4a46-8803-a2642b6220c5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.473221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1387/2745] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:43.477706 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.490506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d99ab-9a5e-4df4-8b11-d1bf6f5d5f84 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.496238 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.497401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.497813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.498202 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] PATH_INFO: `/v3/projects/d34ea1fbde72463b873cdacf7487301b/users/560c0e307180420795cc31b94128fa9c/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.501245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34ea1fbde72463b873cdacf7487301b, user_id=560c0e307180420795cc31b94128fa9c, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.504593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.519218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1359/2746] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:43 2022] PUT /identity/v3/projects/d34ea1fbde72463b873cdacf7487301b/users/560c0e307180420795cc31b94128fa9c/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:43.529896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.530201 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.530495 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.562578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] MFA Rules not processed for user `560c0e307180420795cc31b94128fa9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:43.563881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:43.660972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1388/2747] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:43.673555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.675613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:43.823711 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8379f-5c2f-4898-998a-7dbac92a8baf admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.827561 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.828508 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.829000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.829502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.842164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:43.844259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:43.856350 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1360/2748] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:43.974783 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:43.984885 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134f6444-081b-4046-89f7-c5ea26e0e5dd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:43.988776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:43.989440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:43.989732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:43.990018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:43.996524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:44.058896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:44.060416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966db20d-3432-4168-8a95-7205e2615244 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:44.070269 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1389/2749] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:46.659120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:46.699781 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:46.739665 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fea4e5-0fa5-4fba-bcd3-06cca2ec7cc0 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:46.755896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:46.756461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:46.756749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:46.757054 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] PATH_INFO: `/v3/users/df79ea5f32354aa883645c02c751fec4` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:46.758298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df79ea5f32354aa883645c02c751fec4)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:46.760809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:46.820086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df79ea5f32354aa883645c02c751fec4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:46.820811 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df79ea5f32354aa883645c02c751fec4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:46.833442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df79ea5f32354aa883645c02c751fec4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:46.834093 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df79ea5f32354aa883645c02c751fec4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:46.842237 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df79ea5f32354aa883645c02c751fec4'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:46.858121 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1361/2750] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:46 2022] DELETE /identity/v3/users/df79ea5f32354aa883645c02c751fec4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:46.862808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:46.863325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:46.863426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:46.882203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:46.883425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:46.944929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1390/2751] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:46.955382 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:46.957071 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:47.016197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa9d3-58a6-41a6-ae7f-239c63d56261 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:47.019843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:47.020515 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:47.020808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:47.021091 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:47.030569 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:47.032030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:47.042313 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1362/2752] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:07:47.320304 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:47.321962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:47.334963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b794fa9-b101-438b-9fca-f2b9a6e36e8d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:47.340434 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:47.343142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:47.343441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:47.343726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] PATH_INFO: `/v3/projects/e5fa913bada04fcfae0549a6491a48e6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:47.345109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fa913bada04fcfae0549a6491a48e6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:47.347296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:47.403422 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fa913bada04fcfae0549a6491a48e6', 'request_id': 'req-896177af-eacd-4bf2-be2a-8d6571e70d0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.410443 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fa913bada04fcfae0549a6491a48e6', 'request_id': 'req-896177af-eacd-4bf2-be2a-8d6571e70d0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.417704 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fa913bada04fcfae0549a6491a48e6', 'request_id': 'req-896177af-eacd-4bf2-be2a-8d6571e70d0b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.432751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] The token cache is being invalidate because project e5fa913bada04fcfae0549a6491a48e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:47.433056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.433758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.435320 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1391/2753] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:47 2022] DELETE /identity/v3/projects/e5fa913bada04fcfae0549a6491a48e6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:47.439381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:47.449469 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c802a-8761-4f67-b7b8-493dd637e41b service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:47.506904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:47.515963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:47.517296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:47.517747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:47.518204 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] PATH_INFO: `/v3/users/560c0e307180420795cc31b94128fa9c` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:47.519773 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=560c0e307180420795cc31b94128fa9c)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:47.521725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:47.575062 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '560c0e307180420795cc31b94128fa9c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.575874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '560c0e307180420795cc31b94128fa9c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.588911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '560c0e307180420795cc31b94128fa9c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.589580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '560c0e307180420795cc31b94128fa9c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.596441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b0f14f67-b499-4fb1-aad6-23d4999e8719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '560c0e307180420795cc31b94128fa9c'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:47.610840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1363/2754] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:07:47 2022] DELETE /identity/v3/users/560c0e307180420795cc31b94128fa9c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:47.954835 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:47.956360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:47.970610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896177af-eacd-4bf2-be2a-8d6571e70d0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.011450 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:48.050940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.051836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.052146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.052568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] PATH_INFO: `/v3/projects/d34ea1fbde72463b873cdacf7487301b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.053786 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34ea1fbde72463b873cdacf7487301b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.055751 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.104441 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34ea1fbde72463b873cdacf7487301b', 'request_id': 'req-90f96a11-9e78-491f-bc53-06a6d4f32e85'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:48.111736 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34ea1fbde72463b873cdacf7487301b', 'request_id': 'req-90f96a11-9e78-491f-bc53-06a6d4f32e85'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:48.117571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34ea1fbde72463b873cdacf7487301b', 'request_id': 'req-90f96a11-9e78-491f-bc53-06a6d4f32e85'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:48.132987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] The token cache is being invalidate because project d34ea1fbde72463b873cdacf7487301b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:07:48.133282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:48.133983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-90f96a11-9e78-491f-bc53-06a6d4f32e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:07:48.136083 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1392/2755] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:07:47 2022] DELETE /identity/v3/projects/d34ea1fbde72463b873cdacf7487301b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:48.141103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.141579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.141680 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.162609 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:48.163881 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:48.231393 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1364/2756] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:48.239588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.239976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.240265 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.261479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:48.262863 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:48.271080 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1393/2757] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:48.276986 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.287649 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfc242-3d4c-4de0-a18d-27193ee3c11f None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.292488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.293345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.293863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.294517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.295888 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:07:48.296361 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.298526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.313405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1365/2758] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:07:48.317844 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.328994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e11464-2c11-415e-a096-8472e1f329f7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.333776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.334446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.334732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.335009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.335708 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.338014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.369493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1394/2759] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:07:48 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:07:48.370669 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.381013 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d0867-922b-445d-9dd7-0b15ce5da828 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.442539 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.443214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.443503 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.443904 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.444618 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.446558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.455490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:48.470130 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1366/2760] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:07:48 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:07:48.474496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.485002 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f6286-12b7-4a98-8fb9-2281c301791b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.489984 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.490658 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.491295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.491295 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.492425 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.494771 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.512181 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1395/2761] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:07:48.517048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.528512 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f258f-3a2d-4bc0-9c3c-8bec424a2021 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.533169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.533848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.534142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.534450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.535190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.536835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc172329-3a83-49fd-bb46-961c6d76ceeb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.548181 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1367/2762] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:07:48.552388 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.562663 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c05def-b25c-4a1c-9bd2-46f0944afe7f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.567218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.568128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.568410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.568690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] PATH_INFO: `/v3/projects/c6cfa99adf304f2196703f2623311489/users/cfc1c709af9941e99eaee6b41f89176d/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.582617 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cfa99adf304f2196703f2623311489, user_id=cfc1c709af9941e99eaee6b41f89176d, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.585284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.598518 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1396/2763] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:07:48 2022] PUT /identity/v3/projects/c6cfa99adf304f2196703f2623311489/users/cfc1c709af9941e99eaee6b41f89176d/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:07:48.617217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.617547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.617841 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.645938 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] MFA Rules not processed for user `cfc1c709af9941e99eaee6b41f89176d`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:07:48.647147 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:48.714048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1368/2764] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:07:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:07:48.725147 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:48.726852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:48.793834 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b50b6e-56f6-4777-bd14-90974b1a5a27 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:48.798286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:48.799243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:48.799535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:48.799874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:48.809159 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:48.810619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:48.815493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:07:48.823274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1397/2765] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:07:50.793333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:07:50.794572 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:50.884768 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b05f14-6212-4620-a027-5df2d805f9d0 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:07:50.890490 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:07:50.891145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:07:50.891442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:07:50.891725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:07:50.893548 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:07:50.905776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:07:50.907387 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30cd43e-e534-4d3c-a99d-128885e41038 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:07:50.919281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1369/2766] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:07:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 14:08:00.913624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:INSERT=2,keystone:SELECT=73 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:09.525250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:09.586872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74c4f6-2d62-4d71-8b1b-7ad951c4126f service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:09.591281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:09.592726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:09.593019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:09.593300 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] PATH_INFO: `/v3/users/cfc1c709af9941e99eaee6b41f89176d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:09.594496 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc1c709af9941e99eaee6b41f89176d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:09.596497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:09.642193 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc1c709af9941e99eaee6b41f89176d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:09.648954 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc1c709af9941e99eaee6b41f89176d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:09.661396 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc1c709af9941e99eaee6b41f89176d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:09.662067 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc1c709af9941e99eaee6b41f89176d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:09.662679 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc1c709af9941e99eaee6b41f89176d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:09.675190 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1398/2767] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:08:09 2022] DELETE /identity/v3/users/cfc1c709af9941e99eaee6b41f89176d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:08:09.679834 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:09.680294 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:09.680414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:09.697067 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:09.698267 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:09.766084 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1370/2768] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:09.776061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:09.777532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:09.837274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bf8c09-9530-424f-9328-a8954af4f59e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:09.840972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:09.841552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:09.841877 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:09.842191 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:09.851741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:09.860034 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ad7fe-b8b3-4226-9a17-9f31b905a11e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:09.863619 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1399/2769] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:08:10.127650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.129423 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:10.142996 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a1548-d682-41f7-a799-7a90a552eea2 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.148436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.149491 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.149783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.150089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] PATH_INFO: `/v3/projects/c6cfa99adf304f2196703f2623311489` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.151305 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cfa99adf304f2196703f2623311489)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.153229 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.197758 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cfa99adf304f2196703f2623311489', 'request_id': 'req-b5804ac1-fbad-4522-b909-7d87478e168d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:10.203741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cfa99adf304f2196703f2623311489', 'request_id': 'req-b5804ac1-fbad-4522-b909-7d87478e168d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:10.209137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cfa99adf304f2196703f2623311489', 'request_id': 'req-b5804ac1-fbad-4522-b909-7d87478e168d'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:10.221922 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] The token cache is being invalidate because project c6cfa99adf304f2196703f2623311489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:08:10.222221 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:10.222840 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-b5804ac1-fbad-4522-b909-7d87478e168d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:08:10.224400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1371/2770] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:08:10 2022] DELETE /identity/v3/projects/c6cfa99adf304f2196703f2623311489 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:08:10.229720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.230099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.230455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.258874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:10.260108 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:10.269374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:10.324781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1400/2771] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:10.332634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.332935 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.333226 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.351225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:10.352475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:10.360976 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1372/2772] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:10.365259 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.377240 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f69c4-1e0a-4184-9600-76451e63e2e7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.382327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.383018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.383307 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.383590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.384369 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:08:10.384683 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.386294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.397832 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1401/2773] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:08:10.402086 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.411518 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1e1b6-7589-45dc-aed1-6ceea92bed76 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.416149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.416835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.417122 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.417436 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.418161 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.420613 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.446316 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1373/2774] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:08:10 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:08:10.450230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.460572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d934e-7207-4cd5-a888-812d6021d162 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.517636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.518313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.518601 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.518881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.519590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.521540 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.542513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1402/2775] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:08:10 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:08:10.547015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.557205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65f5c-291c-4743-baa2-772420f96213 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.561728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.562381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.562668 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.562948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.563699 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.565333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.575402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1374/2776] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:08:10.579589 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.589359 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4dd24-3e97-40d3-99fb-8d2cf63f7b90 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.594212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.594861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.595148 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.595429 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.596202 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.597788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5d17e1-d777-407a-a265-ef59b603b855 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.607915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1403/2777] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:08:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:08:10.611932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.621702 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a443b-b7c9-473c-a63e-75ffe249d6bb admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.626747 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.627677 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.627946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.628341 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] PATH_INFO: `/v3/projects/1dfef898db944127a1407a9be98cac4f/users/8dd68e21b78f49b09a958415b17f6aa8/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.642990 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfef898db944127a1407a9be98cac4f, user_id=8dd68e21b78f49b09a958415b17f6aa8, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.645666 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.658483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1375/2778] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:08:10 2022] PUT /identity/v3/projects/1dfef898db944127a1407a9be98cac4f/users/8dd68e21b78f49b09a958415b17f6aa8/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:08:10.663448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffbd8-ec2a-4c1c-b3f5-684abaf3bc27 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.663745 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffbd8-ec2a-4c1c-b3f5-684abaf3bc27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.664063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ffbd8-ec2a-4c1c-b3f5-684abaf3bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.681558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2d5ffbd8-ec2a-4c1c-b3f5-684abaf3bc27 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:10.682763 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2d5ffbd8-ec2a-4c1c-b3f5-684abaf3bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:10.739570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1404/2779] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:10.749653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:10.751588 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:10.773362 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:10.809466 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c6e39f-e536-46e4-b82e-4130553368fa admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:10.813777 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:10.814420 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:10.814709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:10.814997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:10.823903 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:10.825325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:10.835529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1376/2780] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:08:11.532628 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9e58-4d9a-400d-8106-4c652a02c46d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:11.532952 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9e58-4d9a-400d-8106-4c652a02c46d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:11.533263 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f9e58-4d9a-400d-8106-4c652a02c46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:11.562121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-528f9e58-4d9a-400d-8106-4c652a02c46d None None] MFA Rules not processed for user `8dd68e21b78f49b09a958415b17f6aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:11.564222 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-528f9e58-4d9a-400d-8106-4c652a02c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:11.572872 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:11.633571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1405/2781] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:11.645111 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:11.646504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:11.709894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684a913-71f3-4691-b4f8-7fd30f978bd5 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:11.713433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:11.714092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:11.714381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:11.714663 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:11.723577 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:11.725048 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:11.734273 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1377/2782] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:08:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:08:13.080668 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1ac2-c723-40fd-88d5-e4ffff091606 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:13.081195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1ac2-c723-40fd-88d5-e4ffff091606 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:13.081297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1ac2-c723-40fd-88d5-e4ffff091606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:13.098325 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8d3b1ac2-c723-40fd-88d5-e4ffff091606 None None] MFA Rules not processed for user `8dd68e21b78f49b09a958415b17f6aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:08:13.099513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d3b1ac2-c723-40fd-88d5-e4ffff091606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:13.106206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1406/2783] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:08:21.729943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=43 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:23.093284 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=26 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:08:41.498641 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:08:41.500140 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:41.591019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69b5ec-5a9e-480f-b711-8dd43e829bb2 service cinder] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:08:41.596283 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:08:41.596957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:08:41.597243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:08:41.597524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:08:41.599258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:08:41.610610 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:08:41.612215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:08:41.622002 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1378/2784] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:08:51.617786 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=41 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:09:30.340051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:09:30.340560 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:09:30.340673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:09:30.359017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] MFA Rules not processed for user `8dd68e21b78f49b09a958415b17f6aa8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:09:30.360169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:09:30.367476 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1407/2785] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:09:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:09:30.809622 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:09:30.811018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:09:30.822475 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c541dfe4-97ce-436d-bbcb-c2c3f99489a3 service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:09:30.826009 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:09:30.826633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:09:30.826948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:09:30.827239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:09:30.829000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:09:30.838430 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:09:30.839868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:09:30.847695 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1379/2786] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:09:40.352732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:09:40.844206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:23.256013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:23.265583 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fccda-dc3a-4fee-8ecc-e54d06739ccb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:23.269291 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:23.269929 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:23.270211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:23.270486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:23.272297 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:23.330989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:23.332544 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:23.342598 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1408/2787] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:30.560634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:30.569770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26307b6-c70f-4cb5-bf9d-542de8c4db62 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:30.574011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:30.574648 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:30.574930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:30.575209 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:30.577020 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:30.588560 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:30.589974 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:30.599335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1380/2788] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:33.337288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=28 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:33.647587 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:33.649050 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:33.664519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce1489-5358-4bb3-a7fb-83acdbcbfa5f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:33.669914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:33.670578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:33.670900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:33.671184 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:33.673043 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:33.685362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:33.686951 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:33.700038 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1409/2789] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:36.859298 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:36.860894 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:36.877823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae451fd-b843-4019-b585-a53a0846f446 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:36.882759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:36.883628 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:36.883946 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:36.884756 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] PATH_INFO: `/v3/users/72aba3691e2e4b40b6643c8318caa9d6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:36.885954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72aba3691e2e4b40b6643c8318caa9d6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:36.887880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:36.945325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72aba3691e2e4b40b6643c8318caa9d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:36.945963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72aba3691e2e4b40b6643c8318caa9d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:36.959101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72aba3691e2e4b40b6643c8318caa9d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:36.959765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72aba3691e2e4b40b6643c8318caa9d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:36.966900 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72aba3691e2e4b40b6643c8318caa9d6'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:36.981435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1381/2790] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:10:36 2022] DELETE /identity/v3/users/72aba3691e2e4b40b6643c8318caa9d6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:37.207738 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:37.222116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48751b-66fd-47b1-8533-cca917bea7d0 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:37.285044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:37.285914 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.286205 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.286488 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] PATH_INFO: `/v3/projects/cbe0605c62d04bed8edf14047746e765` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.287685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe0605c62d04bed8edf14047746e765)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:37.289661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:37.339517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe0605c62d04bed8edf14047746e765', 'request_id': 'req-31b22574-9e19-4c36-8241-27c6581b2c28'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.346243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe0605c62d04bed8edf14047746e765', 'request_id': 'req-31b22574-9e19-4c36-8241-27c6581b2c28'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.352105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe0605c62d04bed8edf14047746e765', 'request_id': 'req-31b22574-9e19-4c36-8241-27c6581b2c28'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.365900 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] The token cache is being invalidate because project cbe0605c62d04bed8edf14047746e765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:10:37.366218 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.366850 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.368426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1410/2791] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:10:37 2022] DELETE /identity/v3/projects/cbe0605c62d04bed8edf14047746e765 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:37.372562 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:37.374169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:37.389262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5d747-5609-46f4-92ea-c1c7c422eba6 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:37.415052 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:37.417253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:37.456374 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:37.457175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.457465 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.457748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] PATH_INFO: `/v3/users/232a52cf45c740cbb64fb25a15be9f22` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.458939 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] RBAC: Authorizing `identity:delete_user(user_id=232a52cf45c740cbb64fb25a15be9f22)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:37.460889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:37.488714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b22574-9e19-4c36-8241-27c6581b2c28 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:37.493969 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:37.495062 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.495522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.495937 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] PATH_INFO: `/v3/users/8dd68e21b78f49b09a958415b17f6aa8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.497439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd68e21b78f49b09a958415b17f6aa8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:37.499843 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:37.516185 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '232a52cf45c740cbb64fb25a15be9f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.516858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '232a52cf45c740cbb64fb25a15be9f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.522309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:37.529118 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:37.531635 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '232a52cf45c740cbb64fb25a15be9f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.532318 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '232a52cf45c740cbb64fb25a15be9f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.543391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '232a52cf45c740cbb64fb25a15be9f22'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.552651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd68e21b78f49b09a958415b17f6aa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.559437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1382/2792] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:10:37 2022] DELETE /identity/v3/users/232a52cf45c740cbb64fb25a15be9f22 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:37.560891 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd68e21b78f49b09a958415b17f6aa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.574410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd68e21b78f49b09a958415b17f6aa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.575363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd68e21b78f49b09a958415b17f6aa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.576024 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd68e21b78f49b09a958415b17f6aa8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.589934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1411/2793] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:10:37 2022] DELETE /identity/v3/users/8dd68e21b78f49b09a958415b17f6aa8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:10:37.732766 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:37.734244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:37.776379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:37.778538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:37.815835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c30ba9-1c87-45f5-96ca-18e4999f770d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:37.847855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df9e0b-1184-418b-9bee-fcd6148f1f7a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:37.852416 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:37.853847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.854133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.854410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] PATH_INFO: `/v3/projects/1dfef898db944127a1407a9be98cac4f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.855608 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfef898db944127a1407a9be98cac4f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:37.857554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:37.874966 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:37.875847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.876146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.876431 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] PATH_INFO: `/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.877636 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a24bbb97ea3f4c1684ae57b2d2314740)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:37.879850 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:37.905614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfef898db944127a1407a9be98cac4f', 'request_id': 'req-827866cd-d9c0-4ab9-869f-24b6b88941b4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.912436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfef898db944127a1407a9be98cac4f', 'request_id': 'req-827866cd-d9c0-4ab9-869f-24b6b88941b4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.918712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfef898db944127a1407a9be98cac4f', 'request_id': 'req-827866cd-d9c0-4ab9-869f-24b6b88941b4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.930551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a24bbb97ea3f4c1684ae57b2d2314740', 'request_id': 'req-695cf31b-ed15-401b-97c8-368e08b5daa1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.932367 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] The token cache is being invalidate because project 1dfef898db944127a1407a9be98cac4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:10:37.932661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.933310 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-827866cd-d9c0-4ab9-869f-24b6b88941b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.935098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1412/2794] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:10:37 2022] DELETE /identity/v3/projects/1dfef898db944127a1407a9be98cac4f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:37.937774 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a24bbb97ea3f4c1684ae57b2d2314740', 'request_id': 'req-695cf31b-ed15-401b-97c8-368e08b5daa1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.940088 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:37.940384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:37.940665 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:37.943620 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a24bbb97ea3f4c1684ae57b2d2314740', 'request_id': 'req-695cf31b-ed15-401b-97c8-368e08b5daa1'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.956952 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] The token cache is being invalidate because project a24bbb97ea3f4c1684ae57b2d2314740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:10:37.957439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.958076 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-695cf31b-ed15-401b-97c8-368e08b5daa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:10:37.959501 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:37.959783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1383/2795] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:10:37 2022] DELETE /identity/v3/projects/a24bbb97ea3f4c1684ae57b2d2314740 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:37.960767 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:37.984028 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:38.034892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1413/2796] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:10:38.036724 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.037022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.037308 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.057489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:38.058743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:38.066860 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1384/2797] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:38.071391 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.081823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd8cdb-f119-4ec9-8a42-fcc7e70c1460 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.086719 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.088087 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.088374 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.088649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.089588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:10:38.089903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.091532 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.108530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1414/2798] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:10:38.110079 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.119723 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b29553-7b5a-48d2-b086-18393c0731a6 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.124439 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.125126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.125415 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.125696 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.126412 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.128381 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.133281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:38.164358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1385/2799] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:10:38 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:10:38.165522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.176022 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823005d-3b70-41a0-aeb4-9e06d5d8d8e1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.234105 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.234759 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.234958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.235236 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.235971 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.237892 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.258211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1415/2800] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:10:38 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:10:38.262954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.273274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c839c-0bb9-42b1-bf35-6c20987ded28 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.278506 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.279164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.279449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.279727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.280647 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.282266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.292321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1386/2801] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:10:38.296234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.305871 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe19cb9-8a70-47bf-93d1-3edae15d684e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.310457 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.311163 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.311506 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.311847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.312612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.314282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7b972-f6ef-49c6-ba1b-90b709b56686 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.325175 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1416/2802] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:10:38.329049 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.339297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9d3b8-1d9b-4bc1-8106-b3309182dc77 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.343858 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.344719 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.345001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.345279 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] PATH_INFO: `/v3/projects/b878ba5622b445a0be6f66811572d4cd/users/a21f7d97b64242cfa8bebe82b728c8bd/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.359848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b878ba5622b445a0be6f66811572d4cd, user_id=a21f7d97b64242cfa8bebe82b728c8bd, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.362835 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.375314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1387/2803] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:10:38 2022] PUT /identity/v3/projects/b878ba5622b445a0be6f66811572d4cd/users/a21f7d97b64242cfa8bebe82b728c8bd/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:38.380893 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c2760-7c25-48f4-af3f-26eb1d3d9a49 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.381219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c2760-7c25-48f4-af3f-26eb1d3d9a49 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.381547 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c2760-7c25-48f4-af3f-26eb1d3d9a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.398858 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1e1c2760-7c25-48f4-af3f-26eb1d3d9a49 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:38.400677 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1e1c2760-7c25-48f4-af3f-26eb1d3d9a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:38.459065 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1417/2804] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:38.468504 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:38.470222 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:38.541070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de097bf3-d358-47d8-b39d-16973ef6187c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:38.544741 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:38.545386 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:38.545671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:38.545948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:38.554709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:38.556765 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:38.566286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1388/2805] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:42.579467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:42.579987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:42.580075 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:42.616395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] MFA Rules not processed for user `a21f7d97b64242cfa8bebe82b728c8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:42.617693 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:42.653941 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:42.705149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1418/2806] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:42.709166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:42.720525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3f109-46ca-490c-97a4-6086e1bd11e1 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:42.725847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:42.726608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:42.726961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:42.727310 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:42.728634 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:42.731190 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:42.766224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1389/2807] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:10:42 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:10:42.767395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:42.787192 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ad1d8-bcc2-4b8f-a61e-14188d1c0e9f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:42.862294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:42.862991 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:42.863285 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:42.863577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:42.864334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:42.866298 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:42.889846 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1419/2808] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:10:42 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:10:42.893742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:42.904920 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3175f5-5898-4dae-83fb-89ebf9624c46 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:42.909543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:42.910214 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:42.910505 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:42.910789 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:42.911523 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:42.913149 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:42.923865 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1390/2809] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:10:42.927994 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:42.938368 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2327a-4af3-40bf-8230-cf242161db13 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:42.942902 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:42.943778 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:42.944098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:42.944380 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] PATH_INFO: `/v3/projects/1ccf50c82ba14988a342b585696c8245/users/0713a6e92f73429bac98626de063744f/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:42.959639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccf50c82ba14988a342b585696c8245, user_id=0713a6e92f73429bac98626de063744f, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:42.962985 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:42.976545 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1420/2810] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:10:42 2022] PUT /identity/v3/projects/1ccf50c82ba14988a342b585696c8245/users/0713a6e92f73429bac98626de063744f/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:10:42.980743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:42.990745 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cfe62-2610-41dd-a118-262fce04408d admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:43.046886 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:43.047555 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:43.047869 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:43.048154 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:43.048877 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:43.050473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41d4a-5fba-4aaa-aba4-2ef728b2ccda admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:43.061880 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1391/2811] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:10:43.065976 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:43.077021 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00371677-9087-4012-8bb3-79ba6e004f14 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:43.081776 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:43.082686 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:43.082970 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:43.083253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] PATH_INFO: `/v3/projects/1ccf50c82ba14988a342b585696c8245/users/0713a6e92f73429bac98626de063744f/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:43.085288 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccf50c82ba14988a342b585696c8245, user_id=0713a6e92f73429bac98626de063744f, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:43.088519 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:43.099584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1421/2812] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:10:43 2022] PUT /identity/v3/projects/1ccf50c82ba14988a342b585696c8245/users/0713a6e92f73429bac98626de063744f/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:10:48.060815 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e23c-1297-4d0c-9903-306f68e0eee5 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:48.061346 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e23c-1297-4d0c-9903-306f68e0eee5 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:48.061433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473e23c-1297-4d0c-9903-306f68e0eee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:48.092102 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-0473e23c-1297-4d0c-9903-306f68e0eee5 None None] MFA Rules not processed for user `0713a6e92f73429bac98626de063744f`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:48.093335 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-0473e23c-1297-4d0c-9903-306f68e0eee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:48.098579 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:48.170687 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1392/2813] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:48.184119 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:48.185878 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:48.252580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c386cc-6294-4906-9797-46687d1f1ddb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:48.256502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:48.257154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:48.257446 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:48.257732 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:48.289384 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:48.320243 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:48.322405 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:48.331531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1422/2814] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:48.427428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657b22-876e-456b-9186-ce9762cf292d None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:48.427727 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657b22-876e-456b-9186-ce9762cf292d None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:48.428097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f657b22-876e-456b-9186-ce9762cf292d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:48.491981 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-6f657b22-876e-456b-9186-ce9762cf292d None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:48.493233 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-6f657b22-876e-456b-9186-ce9762cf292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:48.570779 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1393/2815] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:48.620438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:48.621884 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:48.713103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55716f1-1cce-4c9a-a66b-7788719133c9 service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:48.718146 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:48.718790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:48.719078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:48.719361 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:48.727352 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:48.729264 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:48.737479 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1423/2816] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:10:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 14:10:57.348207 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:57.348692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:57.348814 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:57.369659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] MFA Rules not processed for user `a21f7d97b64242cfa8bebe82b728c8bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:57.369659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:57.378126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1394/2817] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:10:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:10:57.416839 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:57.436013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7184-5ab7-46e1-a5d5-5d5fe06077db service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:57.439568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb86cbb65b734fc69bf7604414f8c9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:57.440230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:57.440525 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:57.440806 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:57.442614 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:57.528860 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:57.530343 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db913a8-d922-499c-a01f-dc2c8a075c36 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:57.542695 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:57.551718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1424/2818] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:58.084040 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:58.085633 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:58.144141 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7acc86-b11b-4ff6-92e0-1eeadd1a4733 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:58.148018 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:58.148710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:58.149082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:58.149311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:58.151342 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:58.163059 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:10:58.166208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:58.167703 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:58.179940 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1395/2819] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:10:59.235174 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:59.235641 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:59.235980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:59.255128 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:10:59.256360 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:10:59.264462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1425/2820] 10.222.0.24 () {58 vars in 1009 bytes} [Tue May 24 14:10:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:10:59.280514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:10:59.292447 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d461724c-077c-4dda-b9bc-35873bc87fcb service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:10:59.297280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:10:59.297954 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:10:59.298239 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:10:59.298528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:10:59.306827 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:10:59.308449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:10:59.316863 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1396/2821] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:10:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:11:09.249782 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:11:09.313476 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=8 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:11:11.915469 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:11:11.927365 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab80f1f-f2cd-4a27-9f10-c14a4e1f409c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:11:11.930950 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:11:11.931947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:11:11.932357 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:11:11.932638 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:11:11.934426 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:11:11.943980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:11:11.945409 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:11:11.954031 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1426/2822] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:11:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:11:21.950486 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:11:30.511941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:11:30.521563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf6e49-6477-4bf0-b72f-db652147ced5 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:11:30.525133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:11:30.525776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:11:30.526068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:11:30.526529 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:11:30.535520 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:11:30.537026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:11:30.547281 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1397/2823] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:11:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:11:40.543089 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:13:09.327524 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:09.328980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:09.406149 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173c84be-d8ed-4645-ac41-a89b9b5a8b8f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:09.409689 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:09.410335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:09.410624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:09.410904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:09.412685 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:09.470801 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:09.472493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b8d0b-2504-4eef-ba38-f01862f70c41 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:09.482655 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1427/2824] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:11.951989 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:11.953474 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:11.973848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a5145-1e8a-4c0d-8f45-274cfec8cc6e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:11.982268 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:11.982932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:11.983218 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:11.983499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:11.985307 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:12.000162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:12.001813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:12.014868 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1398/2825] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:15.383271 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:15.383790 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:15.383940 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:15.401015 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] MFA Rules not processed for user `54fce865bd6842c19583b3bfe607fc32`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:13:15.402351 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:15.409142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1428/2826] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:13:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:13:15.710097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:15.719949 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177e85a-d973-4fec-acaa-1526e05c1fb7 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:15.723438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:15.724109 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:15.724400 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:15.724684 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:15.726427 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:15.736451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:15.737874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:15.746530 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1399/2827] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:16.058567 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:16.060011 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:16.136154 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ac161-9c2c-48ad-8873-ec7115125416 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:16.144484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:16.145145 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:16.145440 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:16.145724 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:16.147517 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:16.159855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:16.161464 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:16.171522 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1429/2828] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:24.553030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:24.554587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:24.571832 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce015ee-766e-4dc8-87b2-8808ccfd9085 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:24.577632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:24.578287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:24.578580 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:24.578866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:24.580656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:24.593611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:24.595194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:24.607038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1400/2829] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:26.167649 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:13:28.800523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:28.819487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f737999-8ce2-45ab-b3bb-669d26c0b0b0 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:28.825949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bb056b4578442038d33d4fdf9de9584', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:28.826610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:28.826904 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:28.827346 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:28.829618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:28.849410 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:28.851590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90adb0-da43-45ed-9c98-d0bb986e5acd service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:28.867621 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1430/2830] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:34.601306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=18 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:13:38.861734 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:13:46.462097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:46.474037 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021eb5c-5317-4316-9a89-d3c247892eb4 service nova] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:46.478916 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:46.479556 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:46.479876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:46.480166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:46.482692 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:46.494158 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:46.495767 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:46.506743 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1401/2831] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:13:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:13:56.501918 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=6 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:13:58.487323 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:58.487788 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:58.487922 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:58.507023 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] MFA Rules not processed for user `6d80ba86a9bf4219a509d11f4617930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:13:58.508281 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:13:58.515762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1431/2832] 10.222.0.24 () {58 vars in 967 bytes} [Tue May 24 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:13:58.555742 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:13:58.568123 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f46bc-6ee2-47d2-8e0e-f5f57f06f7ea service placement] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:13:58.573007 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0bf046435a84f7592fd022c03eda0a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:13:58.573659 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:13:58.573948 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:13:58.574232 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:13:58.582372 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:13:58.583984 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:13:58.592943 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1402/2833] 10.222.0.24 () {60 vars in 1384 bytes} [Tue May 24 14:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 14:14:05.057504 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:14:05.070630 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52f8cd-ae2a-4bbe-967c-aedba1c277dc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:14:05.076956 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:14:05.077640 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:14:05.077924 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:14:05.078204 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:14:05.080116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:14:05.093829 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:14:05.095358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:14:05.114855 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1432/2834] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:14:05.131031 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:14:05.197797 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efed92-90a1-44e3-b324-33741ce06526 service swift] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:14:05.202930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:14:05.203804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:14:05.204126 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:14:05.204410 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] PATH_INFO: `/v3/users/a21f7d97b64242cfa8bebe82b728c8bd` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:14:05.205602 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21f7d97b64242cfa8bebe82b728c8bd)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:14:05.207640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:14:05.253816 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21f7d97b64242cfa8bebe82b728c8bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.254486 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21f7d97b64242cfa8bebe82b728c8bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.266485 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21f7d97b64242cfa8bebe82b728c8bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.267073 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21f7d97b64242cfa8bebe82b728c8bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.273843 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21f7d97b64242cfa8bebe82b728c8bd'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.292852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1403/2835] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:14:05 2022] DELETE /identity/v3/users/a21f7d97b64242cfa8bebe82b728c8bd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:14:05.465804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:14:05.478897 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c2a25-51d6-401c-8d17-24dd366f8897 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:14:05.542579 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:14:05.544200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:14:05.544592 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:14:05.544876 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] PATH_INFO: `/v3/projects/b878ba5622b445a0be6f66811572d4cd` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:14:05.546063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b878ba5622b445a0be6f66811572d4cd)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:14:05.548030 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:14:05.593460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b878ba5622b445a0be6f66811572d4cd', 'request_id': 'req-721b30c6-6612-44b1-a6af-3f72ece5b089'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.599494 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b878ba5622b445a0be6f66811572d4cd', 'request_id': 'req-721b30c6-6612-44b1-a6af-3f72ece5b089'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.605051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b878ba5622b445a0be6f66811572d4cd', 'request_id': 'req-721b30c6-6612-44b1-a6af-3f72ece5b089'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.618086 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] The token cache is being invalidate because project b878ba5622b445a0be6f66811572d4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:14:05.618379 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.619017 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.620577 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1433/2836] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:14:05 2022] DELETE /identity/v3/projects/b878ba5622b445a0be6f66811572d4cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:14:05.624764 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:14:05.626383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:14:05.641021 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07793ae5-ac34-4788-a626-2d77547bf48b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:14:05.697065 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:14:05.697941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:14:05.698248 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:14:05.698511 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] PATH_INFO: `/v3/users/0713a6e92f73429bac98626de063744f` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:14:05.700537 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0713a6e92f73429bac98626de063744f)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:14:05.702446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:14:05.707712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:14:05.749224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0713a6e92f73429bac98626de063744f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.749818 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0713a6e92f73429bac98626de063744f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.762332 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0713a6e92f73429bac98626de063744f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.763006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0713a6e92f73429bac98626de063744f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.770082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0713a6e92f73429bac98626de063744f'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:05.783446 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1404/2837] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:14:05 2022] DELETE /identity/v3/users/0713a6e92f73429bac98626de063744f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:14:05.951334 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:14:05.952791 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:14:05.966672 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b30c6-6612-44b1-a6af-3f72ece5b089 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:14:06.023565 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:14:06.024470 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:14:06.024760 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:14:06.025044 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] PATH_INFO: `/v3/projects/1ccf50c82ba14988a342b585696c8245` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:14:06.026233 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccf50c82ba14988a342b585696c8245)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:14:06.028423 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:14:06.039521 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:14:06.074331 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccf50c82ba14988a342b585696c8245', 'request_id': 'req-3f950009-c11e-45b8-b894-49be1de41ba0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:06.080698 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccf50c82ba14988a342b585696c8245', 'request_id': 'req-3f950009-c11e-45b8-b894-49be1de41ba0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:06.086200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccf50c82ba14988a342b585696c8245', 'request_id': 'req-3f950009-c11e-45b8-b894-49be1de41ba0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:06.099060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] The token cache is being invalidate because project 1ccf50c82ba14988a342b585696c8245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:14:06.099356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:06.100019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:14:06.101597 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1434/2838] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:14:05 2022] DELETE /identity/v3/projects/1ccf50c82ba14988a342b585696c8245 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:14:15.780082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:14:16.096143 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:13.322245 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:13.323792 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:13.380487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5820821-9f3e-4fbc-883e-414768f520fd admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:13.428591 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:13.429242 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:13.429528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:13.429813 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:13.431612 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:13.493117 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:13.494554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:13.505592 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1405/2839] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:16.563130 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:16.565414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:16.580341 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f950009-c11e-45b8-b894-49be1de41ba0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:16.585678 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:16.586538 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:16.586822 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:16.587099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] PATH_INFO: `/v3/users/54fce865bd6842c19583b3bfe607fc32` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:16.588530 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=54fce865bd6842c19583b3bfe607fc32)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:16.590907 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:16.642980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54fce865bd6842c19583b3bfe607fc32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.649942 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54fce865bd6842c19583b3bfe607fc32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.662208 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54fce865bd6842c19583b3bfe607fc32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.662888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54fce865bd6842c19583b3bfe607fc32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.663490 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b300a407-123d-4a15-b6c1-38ebdb479a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54fce865bd6842c19583b3bfe607fc32'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.675752 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1435/2840] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:16 2022] DELETE /identity/v3/users/54fce865bd6842c19583b3bfe607fc32 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:16.855322 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:16.868129 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53770f-699a-4da1-a9be-d4ec75b001a3 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:16.926611 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:16.927401 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:16.927694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:16.928011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] PATH_INFO: `/v3/projects/e436feb329204699a78d749db135acce` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:16.929227 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e436feb329204699a78d749db135acce)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:16.931144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:16.939710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:16.978631 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e436feb329204699a78d749db135acce', 'request_id': 'req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.985276 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e436feb329204699a78d749db135acce', 'request_id': 'req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:16.990712 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e436feb329204699a78d749db135acce', 'request_id': 'req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:17.003438 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] The token cache is being invalidate because project e436feb329204699a78d749db135acce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:17.003730 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:17.004422 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-fb9cadcc-ff6b-4e60-a6ef-4a0c246429bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:17.005945 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1406/2841] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:16 2022] DELETE /identity/v3/projects/e436feb329204699a78d749db135acce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:17.011467 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.011799 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.012178 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.030710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:17.031928 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:17.090535 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1436/2842] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:17.098418 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.098715 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.099000 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.118019 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:17.119286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:17.132391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1407/2843] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:17.138286 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.150449 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91695b7-ff0b-4351-a4b2-67f19fe3b31f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.155312 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.156004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.156292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.156571 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.157299 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:15:17.157609 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.159200 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.170963 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1437/2844] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:15:17.175205 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.185215 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66eee1-0003-4ee0-bd2e-c9d7e698deda None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.189783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.190435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.190725 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.191006 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.191709 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.193656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.228287 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1408/2845] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:17.233109 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.242720 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff018de1-8b70-42e6-b2e5-a55b419f7b4c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.298424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.299074 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.299358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.299635 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.300364 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.302309 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.309700 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:17.324328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1438/2846] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:17.328587 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.338614 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee38ae-3414-4e41-a949-90de06fd9c73 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.344170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.344955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.345250 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.345536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.346277 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.347936 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.358545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1409/2847] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:17.362456 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.372188 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad9c97-37b9-47d8-af5b-88d56423ae6e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.376681 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.377329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.377612 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.377889 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.378610 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.380235 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8705f-3177-43fb-ad3f-709227bfbb2d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.391739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1439/2848] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:17.395713 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.407461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d701b-87d4-4194-83a4-b19e1ca61f3e admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.416280 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.417760 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.418256 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.418697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] PATH_INFO: `/v3/projects/d44b13dce16a4fbdb98623fd6db52dd6/users/650cd8dfbe28441eb8bc28fd035da7fe/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.437057 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44b13dce16a4fbdb98623fd6db52dd6, user_id=650cd8dfbe28441eb8bc28fd035da7fe, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.440164 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.453733 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1410/2849] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:17 2022] PUT /identity/v3/projects/d44b13dce16a4fbdb98623fd6db52dd6/users/650cd8dfbe28441eb8bc28fd035da7fe/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:17.464896 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.465353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.465639 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.491439 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] MFA Rules not processed for user `650cd8dfbe28441eb8bc28fd035da7fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:17.492714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:17.556432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1440/2850] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:17.561070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.622673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3eae4-2826-4abb-8aa7-2aec0ad36a2b admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.627252 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.627932 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.628223 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.628501 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.629201 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.631081 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.656454 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1411/2851] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:17.660534 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.671957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e1191-94aa-4d6e-8d39-b1c050c5cb08 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.731069 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.731667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.731989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.732279 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.732977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.734948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.754438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1441/2852] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:17.758637 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.769608 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedce1cb-8e18-4c62-8364-8c24a94d5f75 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.774208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.774866 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.775153 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.775435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.776197 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.778627 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.780697 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:17.790325 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1412/2853] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:17.794722 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.804590 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a3454-809e-4e62-8b37-62993e0167af admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.809336 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.810213 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.810497 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.810774 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] PATH_INFO: `/v3/projects/77b128dd96a740b084ec72e4f82db038/users/d08257ff3bac4eb3ad9098fb7a4c8f87/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.825714 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b128dd96a740b084ec72e4f82db038, user_id=d08257ff3bac4eb3ad9098fb7a4c8f87, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.828385 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.842342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1442/2854] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:17 2022] PUT /identity/v3/projects/77b128dd96a740b084ec72e4f82db038/users/d08257ff3bac4eb3ad9098fb7a4c8f87/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:15:17.846254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.856409 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9736e873-5d13-40cf-94e2-446a5feb5e4f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.912783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.913458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.913751 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.914041 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.914776 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.916406 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52c32b-0aef-4670-acc0-1bf9f72dafd0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.926853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1413/2855] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:15:17.930947 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:17.941558 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef132bb-4b0e-46b6-86df-1a3f5a87b465 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:17.946121 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:17.947004 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.947296 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.947575 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] PATH_INFO: `/v3/projects/77b128dd96a740b084ec72e4f82db038/users/d08257ff3bac4eb3ad9098fb7a4c8f87/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:17.949887 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b128dd96a740b084ec72e4f82db038, user_id=d08257ff3bac4eb3ad9098fb7a4c8f87, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:17.952556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:17.962785 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1443/2856] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:17 2022] PUT /identity/v3/projects/77b128dd96a740b084ec72e4f82db038/users/d08257ff3bac4eb3ad9098fb7a4c8f87/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:17.974206 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def789b9-b366-4943-b626-89ba46885221 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:17.974502 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def789b9-b366-4943-b626-89ba46885221 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:17.974784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def789b9-b366-4943-b626-89ba46885221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:18.002001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-def789b9-b366-4943-b626-89ba46885221 None None] MFA Rules not processed for user `d08257ff3bac4eb3ad9098fb7a4c8f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:18.003217 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-def789b9-b366-4943-b626-89ba46885221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:18.072937 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1414/2857] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:15:18.084294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:18.086018 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:18.109292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:18.144315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f91efa-68bc-4fb1-8f99-1d5b8d1627c3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:18.147958 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:18.148599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:18.148888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:18.149173 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:18.158543 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:18.160009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:18.169851 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1444/2858] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:18.241016 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def789b9-b366-4943-b626-89ba46885221 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:18.251144 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def789b9-b366-4943-b626-89ba46885221 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:18.254801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:18.255383 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:18.255673 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:18.255997 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:18.312993 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:18.314445 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590366e-1927-46f7-8b12-14a406b7c84b service cinder] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:18.325219 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1415/2859] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:27.255742 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:27.317478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70090429-e914-4197-bbaa-2574f4fd00b2 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:27.321993 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:27.322856 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:27.323153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:27.323552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] PATH_INFO: `/v3/users/650cd8dfbe28441eb8bc28fd035da7fe` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:27.324927 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] RBAC: Authorizing `identity:delete_user(user_id=650cd8dfbe28441eb8bc28fd035da7fe)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:27.326847 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:27.372164 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '650cd8dfbe28441eb8bc28fd035da7fe'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.378894 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '650cd8dfbe28441eb8bc28fd035da7fe'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.391152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '650cd8dfbe28441eb8bc28fd035da7fe'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.391874 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '650cd8dfbe28441eb8bc28fd035da7fe'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.392513 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '650cd8dfbe28441eb8bc28fd035da7fe'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.405983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1445/2860] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:27 2022] DELETE /identity/v3/users/650cd8dfbe28441eb8bc28fd035da7fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:27.410487 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:27.410982 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:27.411097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:27.427691 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:27.428923 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:27.440517 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=100 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:27.489433 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1416/2861] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 14:15:27.499241 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:27.500726 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:27.549133 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:27.563934 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a820d-6511-4198-92c5-757ae9dc627a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:27.568962 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:27.569803 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:27.570139 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:27.570502 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:27.583493 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:27.585007 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:27.595155 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1446/2862] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:27.859146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:27.860590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:27.876038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04fa14-ce81-4d1d-a048-4a6ab75fc818 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:27.881977 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:27.882859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:27.883156 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:27.883441 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] PATH_INFO: `/v3/projects/d44b13dce16a4fbdb98623fd6db52dd6` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:27.884853 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44b13dce16a4fbdb98623fd6db52dd6)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:27.887534 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:27.933576 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44b13dce16a4fbdb98623fd6db52dd6', 'request_id': 'req-cce11221-064b-409f-a5e3-078b308d5a93'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.939755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44b13dce16a4fbdb98623fd6db52dd6', 'request_id': 'req-cce11221-064b-409f-a5e3-078b308d5a93'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.945533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44b13dce16a4fbdb98623fd6db52dd6', 'request_id': 'req-cce11221-064b-409f-a5e3-078b308d5a93'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.958329 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] The token cache is being invalidate because project d44b13dce16a4fbdb98623fd6db52dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:27.958623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.959254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:27.960819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1417/2863] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:27 2022] DELETE /identity/v3/projects/d44b13dce16a4fbdb98623fd6db52dd6 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:27.964966 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:27.974564 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd81a3-ef65-4bb9-b921-17a80a462fd2 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.030342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.031219 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.031503 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.031836 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] PATH_INFO: `/v3/users/d08257ff3bac4eb3ad9098fb7a4c8f87` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.033049 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08257ff3bac4eb3ad9098fb7a4c8f87)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.035013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.081756 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08257ff3bac4eb3ad9098fb7a4c8f87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.088477 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08257ff3bac4eb3ad9098fb7a4c8f87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.100338 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08257ff3bac4eb3ad9098fb7a4c8f87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.101009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08257ff3bac4eb3ad9098fb7a4c8f87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.101629 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-477fe613-a853-47e9-961f-bbed4097333c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08257ff3bac4eb3ad9098fb7a4c8f87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.114056 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1447/2864] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:27 2022] DELETE /identity/v3/users/d08257ff3bac4eb3ad9098fb7a4c8f87 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:28.375531 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.377035 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:28.391671 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce11221-064b-409f-a5e3-078b308d5a93 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.451299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.452259 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.452545 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.452819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] PATH_INFO: `/v3/projects/77b128dd96a740b084ec72e4f82db038` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.453980 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b128dd96a740b084ec72e4f82db038)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.455955 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.497748 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:28.503849 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b128dd96a740b084ec72e4f82db038', 'request_id': 'req-58bafc9e-d2df-4067-99a2-1f05f638cb6e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.509694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b128dd96a740b084ec72e4f82db038', 'request_id': 'req-58bafc9e-d2df-4067-99a2-1f05f638cb6e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.515638 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b128dd96a740b084ec72e4f82db038', 'request_id': 'req-58bafc9e-d2df-4067-99a2-1f05f638cb6e'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.528906 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] The token cache is being invalidate because project 77b128dd96a740b084ec72e4f82db038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80166) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:28.529192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.529802 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-58bafc9e-d2df-4067-99a2-1f05f638cb6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:28.531338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1418/2865] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:28 2022] DELETE /identity/v3/projects/77b128dd96a740b084ec72e4f82db038 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:28.536390 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.536886 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.536996 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.555311 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:28.556552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:28.619625 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1448/2866] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:28.627408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.627690 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.628011 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.646558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:28.647772 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:28.656338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1419/2867] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:28.660546 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.666600 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:28.671329 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41801373-3372-4a4a-a023-cf9d8235c65d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.677099 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.677898 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.678353 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.678808 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.679539 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:15:28.679879 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.681584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.693115 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1449/2868] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:15:28.697442 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.706793 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2ce61a-0ff8-45e1-9f98-cf9caa674acf None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.711547 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.712243 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.712535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.712819 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.713524 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.715962 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.741389 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1420/2869] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:28 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:28.745448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.755194 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626860-5933-415b-a90e-a7deddf84c56 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.809588 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.810250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.810536 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.810812 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.811487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.813432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.831781 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1450/2870] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:28 2022] POST /identity/v3/users => generated 294 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:28.835950 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.846567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f9a8f-bacc-4451-a4df-eb5848ae0b55 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.851163 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.851848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.852133 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.852414 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.853142 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.854808 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.865038 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1421/2871] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:28.868773 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.880248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7838d3-33b8-44bd-9dc0-d79974f7f3a0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.885129 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.886362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.886662 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.887358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.888142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.890048 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6ce3b-f363-46b6-b2f2-fcf12550a3db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.899948 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1451/2872] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:28.904092 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:28.913979 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e180bef9-59e2-4ea7-9ac7-adc3de1d186c admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:28.919220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:28.920131 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.920451 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.920759 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] PATH_INFO: `/v3/projects/d6e7cf89d31c491bbabcc79d8a61c4d0/users/37effe2e8eae46f1a94494264d069eef/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.935521 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e7cf89d31c491bbabcc79d8a61c4d0, user_id=37effe2e8eae46f1a94494264d069eef, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:28.938175 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:28.950694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1422/2873] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:28 2022] PUT /identity/v3/projects/d6e7cf89d31c491bbabcc79d8a61c4d0/users/37effe2e8eae46f1a94494264d069eef/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:28.962078 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:28.962313 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:28.962596 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:28.990066 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] MFA Rules not processed for user `37effe2e8eae46f1a94494264d069eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:28.991253 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:29.054144 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1452/2874] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:29.059105 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.119015 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb692aa-5c93-4e47-836b-37af9f6c4124 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.124244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.124925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.125220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.125500 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] PATH_INFO: `/v3/projects` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.126180 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.128090 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.153061 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1423/2875] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:29 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:29.158852 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.169968 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4aa22-f70c-49ad-a347-7b3b1b5001bc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.225712 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.226305 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.226591 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.226866 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.227554 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.229977 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.236478 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:29.251211 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1453/2876] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:29 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:29.255289 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.266066 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f349cf26-2cbe-44cb-9b67-d70cd71bc04f admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.270640 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.271297 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.271593 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.272022 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.272859 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.274489 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.285026 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1424/2877] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 14:15:29.288986 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.299266 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e47e2-0a1e-484c-bbb6-c6c1239ce8a1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.303784 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.304710 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.304980 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.305261 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] PATH_INFO: `/v3/projects/616cdcf6afbb41f49f09f614463f9a99/users/599c57c31b8f42ada567fd365debf7b5/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.320319 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616cdcf6afbb41f49f09f614463f9a99, user_id=599c57c31b8f42ada567fd365debf7b5, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.322961 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.335999 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1454/2878] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:29 2022] PUT /identity/v3/projects/616cdcf6afbb41f49f09f614463f9a99/users/599c57c31b8f42ada567fd365debf7b5/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:29.341051 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.351525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d0a4c-5818-4775-b790-72b518601f94 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.392292 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:29.410203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.410889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.411182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.411473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.412314 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.413914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69750e-bd42-4097-a104-ce546008962c admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.425085 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1425/2879] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:29.429215 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.439605 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f4c7b-d60a-41fb-ae85-e936079d20b9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.445009 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.445935 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.446224 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.446556 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] PATH_INFO: `/v3/projects/616cdcf6afbb41f49f09f614463f9a99/users/599c57c31b8f42ada567fd365debf7b5/roles/25fbe1911e9347e3b1369ba1ec4e04ea` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.448824 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=616cdcf6afbb41f49f09f614463f9a99, user_id=599c57c31b8f42ada567fd365debf7b5, role_id=25fbe1911e9347e3b1369ba1ec4e04ea)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.452257 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.463442 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1455/2880] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:29 2022] PUT /identity/v3/projects/616cdcf6afbb41f49f09f614463f9a99/users/599c57c31b8f42ada567fd365debf7b5/roles/25fbe1911e9347e3b1369ba1ec4e04ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:29.474044 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.474528 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.474656 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.513624 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] MFA Rules not processed for user `599c57c31b8f42ada567fd365debf7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:29.514842 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:29.586632 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1426/2881] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:29.597881 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:29.599746 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:29.656414 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebf892-8c2a-4dc7-96ea-8aaf3622f021 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:29.661140 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:29.662169 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:29.662667 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:29.663153 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:29.673436 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:29.674882 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3863105-5121-4766-a46f-abe681b13cf2 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:29.684740 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1456/2882] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:30.198082 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:30.258650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403345b7-1b72-4fa2-824b-d52e8c62cb40 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:30.263166 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:30.264099 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:30.264391 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:30.264675 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] PATH_INFO: `/v3/users/37effe2e8eae46f1a94494264d069eef` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:30.265862 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37effe2e8eae46f1a94494264d069eef)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:30.267775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:30.313481 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37effe2e8eae46f1a94494264d069eef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.314137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37effe2e8eae46f1a94494264d069eef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.325930 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37effe2e8eae46f1a94494264d069eef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.326567 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37effe2e8eae46f1a94494264d069eef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.327770 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:30.335369 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37effe2e8eae46f1a94494264d069eef'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.348428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1427/2883] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:30 2022] DELETE /identity/v3/users/37effe2e8eae46f1a94494264d069eef => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:30.353059 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:30.353358 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:30.353643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:30.372595 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:30.373690 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:30.432342 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1457/2884] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:30.442224 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:30.443621 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:30.503619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b183e8a-a591-49b4-9e46-f0e2c8dd5311 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:30.507583 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:30.508258 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:30.508546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:30.508823 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:30.518449 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:30.519879 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:30.529653 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1428/2885] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:30.788552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:30.790019 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:30.804459 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603506d-3a83-4519-beba-880ebec84b1a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:30.811268 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:30.812176 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:30.812462 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:30.812741 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] PATH_INFO: `/v3/projects/d6e7cf89d31c491bbabcc79d8a61c4d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:30.813916 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e7cf89d31c491bbabcc79d8a61c4d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:30.815895 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:30.861933 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e7cf89d31c491bbabcc79d8a61c4d0', 'request_id': 'req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.863020 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:30.869913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e7cf89d31c491bbabcc79d8a61c4d0', 'request_id': 'req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.877103 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e7cf89d31c491bbabcc79d8a61c4d0', 'request_id': 'req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.890758 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] The token cache is being invalidate because project d6e7cf89d31c491bbabcc79d8a61c4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:30.891051 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.891684 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:30.893254 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1458/2886] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:30 2022] DELETE /identity/v3/projects/d6e7cf89d31c491bbabcc79d8a61c4d0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:30.897302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:30.907182 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebabeef3-a364-47dd-ba57-1236d655526e service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:30.963309 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:30.964551 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:30.964847 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:30.965135 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] PATH_INFO: `/v3/users/599c57c31b8f42ada567fd365debf7b5` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:30.966364 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=599c57c31b8f42ada567fd365debf7b5)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:30.968379 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.015170 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '599c57c31b8f42ada567fd365debf7b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.015871 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '599c57c31b8f42ada567fd365debf7b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.027623 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '599c57c31b8f42ada567fd365debf7b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.028286 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '599c57c31b8f42ada567fd365debf7b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.034963 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-a03d31e2-0e87-4af7-9dc8-f0ad97481d44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '599c57c31b8f42ada567fd365debf7b5'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.048302 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1429/2887] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:30 2022] DELETE /identity/v3/users/599c57c31b8f42ada567fd365debf7b5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:31.316983 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.318398 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:31.331581 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4a6bd-2dd8-4429-9e61-d51d39d8abba admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.389356 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.390226 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.390510 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.390787 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] PATH_INFO: `/v3/projects/616cdcf6afbb41f49f09f614463f9a99` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.392328 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=616cdcf6afbb41f49f09f614463f9a99)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.394315 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.440123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '616cdcf6afbb41f49f09f614463f9a99', 'request_id': 'req-f41fb2dc-5b08-43ee-b425-1aed30c0d072'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.446093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '616cdcf6afbb41f49f09f614463f9a99', 'request_id': 'req-f41fb2dc-5b08-43ee-b425-1aed30c0d072'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.451578 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '616cdcf6afbb41f49f09f614463f9a99', 'request_id': 'req-f41fb2dc-5b08-43ee-b425-1aed30c0d072'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.464100 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] The token cache is being invalidate because project 616cdcf6afbb41f49f09f614463f9a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:31.464395 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.465008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f41fb2dc-5b08-43ee-b425-1aed30c0d072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:31.466584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1459/2888] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:31 2022] DELETE /identity/v3/projects/616cdcf6afbb41f49f09f614463f9a99 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:31.471563 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.471896 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.472192 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.480338 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:31.492650 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:31.493852 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:31.552184 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1430/2889] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:31.560624 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.560915 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.561234 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.581116 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:31.582303 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:31.590920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1460/2890] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:31.595146 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.605030 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638c30c-55a5-4243-b74a-4ab84ea334ea None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.609585 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.610236 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.610525 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.610805 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.611546 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:15:31.611889 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.613473 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.625755 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1431/2891] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:15:31.629820 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.639047 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66c7b4-9f8a-4649-8dcf-5e1a8dd88279 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.643580 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.644260 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.644593 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.644818 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.645515 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.647387 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.673237 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1461/2892] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:31 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:31.677311 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.687188 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a022-412c-49d9-ade7-f309f6d45249 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.742240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.742915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.743208 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.743483 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.744220 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.746225 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.766735 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1432/2893] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:31 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 14:15:31.770975 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.782135 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1ac23-28c6-4c62-9cd9-3867776c58a5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.786729 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.787389 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.787680 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.787992 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.788731 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.790335 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.801430 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1462/2894] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:31.805437 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.815536 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904bffc4-969c-44df-b011-d612b5a0ad46 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.820145 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.820803 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.821097 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.821405 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.822169 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.823761 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05959b-2011-47c6-bb5b-8e3c7e08a7a0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.833558 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1433/2895] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:31.837460 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:31.847811 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594ba8c-2f64-49fd-973d-c565a175cd5b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:31.852363 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:31.853250 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.853537 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.853817 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] PATH_INFO: `/v3/projects/c9544261df974ccfb31defe98be27ceb/users/8ad423fceb3a4a7f947c7b3a95e181a3/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.868435 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9544261df974ccfb31defe98be27ceb, user_id=8ad423fceb3a4a7f947c7b3a95e181a3, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:31.871061 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:31.883974 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1463/2896] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:31 2022] PUT /identity/v3/projects/c9544261df974ccfb31defe98be27ceb/users/8ad423fceb3a4a7f947c7b3a95e181a3/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:31.895417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:31.895643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:31.895961 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:31.923461 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] MFA Rules not processed for user `8ad423fceb3a4a7f947c7b3a95e181a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:31.924729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:31.984553 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:31.988417 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1434/2897] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:31.999432 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.001170 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.008132 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:32.062275 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d124a-5fb9-43b6-9b09-498e30fa93ba admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.065957 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.066636 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.066925 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.067206 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.076198 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.077673 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc81d8-3eb6-4596-b68c-78b762de578d service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:32.086643 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1464/2898] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:32.107296 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.169321 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d08a8-9e5b-429e-bfd0-10e2e00b88cc None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.174898 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.175783 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.176101 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.176384 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] PATH_INFO: `/v3/users/8ad423fceb3a4a7f947c7b3a95e181a3` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.177590 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ad423fceb3a4a7f947c7b3a95e181a3)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.179526 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:32.227070 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ad423fceb3a4a7f947c7b3a95e181a3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.227729 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ad423fceb3a4a7f947c7b3a95e181a3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.239499 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ad423fceb3a4a7f947c7b3a95e181a3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.240157 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ad423fceb3a4a7f947c7b3a95e181a3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.247103 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ad423fceb3a4a7f947c7b3a95e181a3'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.260340 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1435/2899] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:32 2022] DELETE /identity/v3/users/8ad423fceb3a4a7f947c7b3a95e181a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:15:32.265568 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.265861 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.266142 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.283531 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:32.284755 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.341063 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1465/2900] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:32.350514 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.351972 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.410999 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc6343-b11f-49f7-bf1a-b7d6fae71fa7 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.414601 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.415244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.415533 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.415809 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.425203 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.426564 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52eb1b-f7bc-4724-8654-8bc2e7131130 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:32.436540 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1436/2901] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:32.712888 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.714584 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.726424 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae65fa-df8e-4203-acae-cd4e55406a5a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.731557 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.732448 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.732739 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.733014 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] PATH_INFO: `/v3/projects/c9544261df974ccfb31defe98be27ceb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.734195 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9544261df974ccfb31defe98be27ceb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.736294 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:32.787292 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9544261df974ccfb31defe98be27ceb', 'request_id': 'req-62148fc7-3b77-45f0-ba44-97b33394c58d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.795572 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9544261df974ccfb31defe98be27ceb', 'request_id': 'req-62148fc7-3b77-45f0-ba44-97b33394c58d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.801230 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9544261df974ccfb31defe98be27ceb', 'request_id': 'req-62148fc7-3b77-45f0-ba44-97b33394c58d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.813804 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] The token cache is being invalidate because project c9544261df974ccfb31defe98be27ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:32.814098 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.814718 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-62148fc7-3b77-45f0-ba44-97b33394c58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:32.816318 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1466/2902] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:32 2022] DELETE /identity/v3/projects/c9544261df974ccfb31defe98be27ceb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:32.821262 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.821775 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.821885 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.841915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:32.844069 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.862848 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:32.906137 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1437/2903] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:32.913795 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.914093 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.914378 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.934016 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:32.935212 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:32.942972 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1467/2904] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:32.946968 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.956855 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee554de7-8731-4fd5-9dbe-22557f3c933a None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.961345 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.962001 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.962290 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.962570 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.963299 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80166) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 14:15:32.963694 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.965358 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:32.976643 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1438/2905] 10.222.0.24 () {58 vars in 1152 bytes} [Tue May 24 14:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 14:15:32.981152 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:32.990060 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26387778-898d-4c63-b185-e4b5118eb36e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:32.994651 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:32.995917 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:32.996203 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:32.996484 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:32.997216 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:32.999107 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.023316 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1468/2906] 10.222.0.24 () {60 vars in 1152 bytes} [Tue May 24 14:15:32 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:33.027498 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:33.037333 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32056bcb-353c-48d1-8e7c-ff2327bb11f0 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:33.094876 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:33.095543 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.095919 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.096212 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] PATH_INFO: `/v3/users` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.096895 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:33.098801 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.118600 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1439/2907] 10.222.0.24 () {60 vars in 1143 bytes} [Tue May 24 14:15:33 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 14:15:33.122848 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:33.128399 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:33.134989 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230cab1-14ee-49ec-ba3a-0e1acdc95507 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:33.141322 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:33.141987 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.142274 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.142550 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.143282 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:33.144906 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.156166 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1469/2908] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:33.160244 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:33.170710 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c2d20-7e57-4711-87c8-f34987a834ff admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:33.175274 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:33.175964 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.176254 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.176535 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] PATH_INFO: `/v3/roles` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.177672 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:33.179460 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4483f1-6b24-4c1c-a774-22166649d07d admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.189458 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1440/2909] 10.222.0.24 () {58 vars in 1121 bytes} [Tue May 24 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 14:15:33.193349 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:33.203248 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e2caa-9846-4ce3-9962-04e6326db3d7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:33.207766 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:33.208616 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.208903 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.209180 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] PATH_INFO: `/v3/projects/03064572bd364d509e4678cd8ab74335/users/d7a43b4a0dc04d7095413378594759b9/roles/b82b050f41064700b9690d91633b2c52` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.224040 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03064572bd364d509e4678cd8ab74335, user_id=d7a43b4a0dc04d7095413378594759b9, role_id=b82b050f41064700b9690d91633b2c52)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:33.226692 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.239949 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1470/2910] 10.222.0.24 () {60 vars in 1482 bytes} [Tue May 24 14:15:33 2022] PUT /identity/v3/projects/03064572bd364d509e4678cd8ab74335/users/d7a43b4a0dc04d7095413378594759b9/roles/b82b050f41064700b9690d91633b2c52 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:33.250488 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] REQUEST_METHOD: `POST` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.250784 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.251068 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.277664 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.auth.core [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] MFA Rules not processed for user `d7a43b4a0dc04d7095413378594759b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80166) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:33.278957 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:33.342408 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1441/2911] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:33.353431 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:33.355807 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:33.411943 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc2664-b8a0-43ea-87fc-072f4525953b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:33.415552 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9da57903126d421b8ac6c0e26b3fb3b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:33.416523 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:33.416823 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:33.417762 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:33.427084 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:33.428570 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e2531-68f9-4fa1-abfc-05cd06a1d978 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:33.438013 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1471/2912] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:43.338120 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:43.433833 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=40,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:15:55.021941 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:55.083240 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5dfa2-b6e7-4795-be49-71406e387c03 None None] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:55.087754 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:55.088644 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:55.088933 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:55.089210 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] PATH_INFO: `/v3/users/d7a43b4a0dc04d7095413378594759b9` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:55.090453 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7a43b4a0dc04d7095413378594759b9)` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:55.092925 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:55.138804 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7a43b4a0dc04d7095413378594759b9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.139450 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7a43b4a0dc04d7095413378594759b9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.151428 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7a43b4a0dc04d7095413378594759b9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.152100 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7a43b4a0dc04d7095413378594759b9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.159554 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.notifications [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7a43b4a0dc04d7095413378594759b9'} {{(pid=80166) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.171928 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1442/2913] 10.222.0.24 () {58 vars in 1223 bytes} [Tue May 24 14:15:55 2022] DELETE /identity/v3/users/d7a43b4a0dc04d7095413378594759b9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 14:15:55.176487 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:55.177008 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:55.177123 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:55.195327 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] MFA Rules not processed for user `a064d7a4fdb343efa665d8a825cdfba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 14:15:55.196661 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:55.257006 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1472/2914] 10.222.0.24 () {58 vars in 957 bytes} [Tue May 24 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 14:15:55.267306 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:55.268749 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:55.324689 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a223-ba6e-4907-b30e-4e55d08223d4 admin admin] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:55.328973 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05e67d8adf624d458533c80507374d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:55.329619 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:55.329911 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:55.330194 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:55.339266 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:55.340728 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:55.350667 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1443/2915] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:15:55.615438 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:15:55.616920 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:15:55.629868 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2a33f-2d62-4d5c-a8c1-72cd7d0f2a27 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:15:55.634733 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a064d7a4fdb343efa665d8a825cdfba7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d5cb275b4fb4682b096f6cd5768d3fe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:15:55.635599 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:15:55.635908 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:15:55.636221 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] PATH_INFO: `/v3/projects/03064572bd364d509e4678cd8ab74335` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:15:55.637618 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=03064572bd364d509e4678cd8ab74335)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:15:55.639631 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:15:55.685676 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03064572bd364d509e4678cd8ab74335', 'request_id': 'req-8b782013-3e4d-44fd-a448-2be93ea6b1fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.691694 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03064572bd364d509e4678cd8ab74335', 'request_id': 'req-8b782013-3e4d-44fd-a448-2be93ea6b1fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.697046 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03064572bd364d509e4678cd8ab74335', 'request_id': 'req-8b782013-3e4d-44fd-a448-2be93ea6b1fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.709455 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] The token cache is being invalidate because project 03064572bd364d509e4678cd8ab74335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 14:15:55.709747 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.710362 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8b782013-3e4d-44fd-a448-2be93ea6b1fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 14:15:55.711913 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1473/2916] 10.222.0.24 () {58 vars in 1232 bytes} [Tue May 24 14:15:55 2022] DELETE /identity/v3/projects/03064572bd364d509e4678cd8ab74335 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 14:16:05.345985 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:16:05.706413 dv-cinder-842129-1-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 14:18:53.551435 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] Authenticating user token {{(pid=80166) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 14:18:53.552914 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:18:53.689323 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9499dce-8415-43e0-9dd8-b50c75741448 service neutron] Validating token access rules against request {{(pid=80166) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 14:18:53.746915 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4121712d68e243f1998602c91d1308d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bff552f050b044b598ad782ed83d42c7', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80166) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 14:18:53.747557 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] REQUEST_METHOD: `GET` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 14:18:53.747874 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] SCRIPT_NAME: `/identity` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 14:18:53.748165 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80166) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 14:18:53.749931 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.fernet_utils [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80166) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 14:18:53.761106 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 14:18:53.762617 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67de15e-985d-44a1-a3c9-94c58efcee9f service placement] RBAC: Authorization granted {{(pid=80166) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 14:18:53.772402 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: [pid: 80166|app: 0|req: 1444/2917] 10.222.0.24 () {60 vars in 1365 bytes} [Tue May 24 14:18:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 14:19:03.768162 dv-cinder-842129-1-iscsi devstack@keystone.service[80166]: DEBUG dbcounter [-] [80166] Writing DB stats keystone:SELECT=89 {{(pid=80166) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}